mCVEs: Using Cross-Scale Collaboration to Support User Interaction with Multiscale Structures

16
Xiaolong Zhang School of Information Resources and Library Science University of Arizona Tucson AZ 85719 USA Correspondence to [email protected] George W. Furnas School of Information University of Michigan Presence, Vol. 14, No. 1, February 2005, 31– 46 © 2005 by the Massachusetts Institute of Technology mCVEs: Using Cross-Scale Collaboration to Support User Interaction with Multiscale Structures Abstract In this paper, a new type of interaction environment, the multiscale collaborative virtual environment (mCVE), is proposed to support multiple users working to- gether at different scale levels. This paper introduces the concept of multiscale col- laboration in the context of 3D virtual environments and describes the benefits of multiscale collaboration for understanding and managing large structures that present important features at different scale levels. After a discussion on the design and implementation of multiscale tools to support the visualization of structures, cross-scale information sharing, and cross-scale action, the paper pre- sents an experimental study showing that cross-scale collaboration can improve user performance. 1 Introduction We live in a world where objects can span many different scale levels. Matter in the real world demonstrates various structures and characteristics at different length-scale levels. For example, images of the Earth at different scales range from a billiard ball in empty space (Figure 1a), to a sphere covered by continents and seas (Figure 1b), to a flat plane with different terrains (Fig- ure 1c). DNA, observed at different scales, could appear as coiled DNA strands (Figure 2a), as the double helix of a DNA strand (Figure 2b), or as molecular building blocks (Figure 2c). These structures all exhibit important characteris- tics at different scale levels, and can be called multiscale structures. Our interests in the world span numerous scales, and cross-scale approaches are often seen. In materials science, for example, scientists need cross-scale structural models, from nanometer to millimeter, to understand the character- istics of materials from the atomic level (e.g., the strength of atomic bonds) to the macroscopic level (e.g., mechanical stress) (Robbins, 2001). In architec- tural design and urban planning, new things need to be built with the consid- eration of larger contexts that embed these things and smaller structures below them (Alexander, Ishikawa, & Silverstein, 1977). Creating a new commercial district may require people to examine how the new district can fit the whole city at the city level, how individual buildings should be designed to match the style of the whole street at the street level, and how changes at one level may affect the design at others (e.g., how opening a new route at the street level Zhang and Furnas 31

Transcript of mCVEs: Using Cross-Scale Collaboration to Support User Interaction with Multiscale Structures

Xiaolong ZhangSchool of Information Resourcesand Library ScienceUniversity of ArizonaTucson AZ 85719 USA

Correspondence toxiaolonguarizonaedu

George W FurnasSchool of InformationUniversity of Michigan

Presence Vol 14 No 1 February 2005 31ndash46

copy 2005 by the Massachusetts Institute of Technology

mCVEs Using Cross-ScaleCollaboration to Support UserInteraction with MultiscaleStructures

Abstract

In this paper a new type of interaction environment the multiscale collaborative

virtual environment (mCVE) is proposed to support multiple users working to-

gether at different scale levels This paper introduces the concept of multiscale col-

laboration in the context of 3D virtual environments and describes the benefits of

multiscale collaboration for understanding and managing large structures that

present important features at different scale levels After a discussion on the

design and implementation of multiscale tools to support the visualization of

structures cross-scale information sharing and cross-scale action the paper pre-

sents an experimental study showing that cross-scale collaboration can improve

user performance

1 Introduction

We live in a world where objects can span many different scale levelsMatter in the real world demonstrates various structures and characteristics atdifferent length-scale levels For example images of the Earth at differentscales range from a billiard ball in empty space (Figure 1a) to a sphere coveredby continents and seas (Figure 1b) to a flat plane with different terrains (Fig-ure 1c) DNA observed at different scales could appear as coiled DNA strands(Figure 2a) as the double helix of a DNA strand (Figure 2b) or as molecularbuilding blocks (Figure 2c) These structures all exhibit important characteris-tics at different scale levels and can be called multiscale structures

Our interests in the world span numerous scales and cross-scale approachesare often seen In materials science for example scientists need cross-scalestructural models from nanometer to millimeter to understand the character-istics of materials from the atomic level (eg the strength of atomic bonds) tothe macroscopic level (eg mechanical stress) (Robbins 2001) In architec-tural design and urban planning new things need to be built with the consid-eration of larger contexts that embed these things and smaller structures belowthem (Alexander Ishikawa amp Silverstein 1977) Creating a new commercialdistrict may require people to examine how the new district can fit the wholecity at the city level how individual buildings should be designed to match thestyle of the whole street at the street level and how changes at one level mayaffect the design at others (eg how opening a new route at the street level

Zhang and Furnas 31

may change the design coherence at the district level)To complete such multiscale tasks we need tools tocontrol the scale level of observation and analysis so thatwe can see structures at different levels and understandtheir relationships This is a challenge given that thenormal interaction scale range of human beings is onlyabout five orders of magnitude from millimeters (103

m) to hundreds of meters (102 m) The size scale rangeof matter in the real world is much larger spanningabout forty-two orders of magnitude from 1016 m to1026 m (Morrison amp Morrison 1994)

We use many different tools in real life to help uscross scale boundaries For example we have micro-scope pictures satellite pictures and scaled model struc-tures for the observation and analysis of objects or struc-tures beyond our normal interaction scale range (egDNA structures architectural structures planet sys-tems) However these artifacts give us only static infor-mation at a particular scale level It is still difficult toreceive information from other scale levels Even whenpictures or model structures at different scales are avail-able interpreting how these pictures or models are re-lated is still a challenge Computer tools for modelingsuffer from similar problems because of the lack of sup-port for multiscale interaction For example Cerius2 avery powerful tool to simulate small and ideal structures(eg unit cells of organic thin films) in materials sci-ence can hardly go beyond the molecular level forlarger structures (eg a structure with 100 100

100 unit cells) that are closer to real-world situations Inarchitectural design complex large-scale projects suchas airports have many thousands of interconnected de-signs at different scales However existing computer-

aided tools are very limited in support of the visualiza-tion organization and navigation of these designs(A Summerfield personal communication December12 2003)

Such a challenge in handling multiscale structures alsoexists when people work with information structures invirtual worlds Information structures such as file sys-tems digital libraries and the World Wide Web are rap-idly becoming larger and larger These structures caneasily have more than tens of thousands of componentsoverwhelming our memories and information-processingcapabilities We usually deal with large structures in ahierarchical way either explicitly or implicitly For ex-ample file systems are organized as explicit hierarchiesThe semantic structure that bonds the maps and relatedgeographic information in Figure 3 uses hierarchy im-plicitly

Hierarchies help us focus on issues within an appro-priate scale level but in limiting our attention to a cer-tain scale range we will lose some important informationof large structures Either detailed content or globalcontext information has to be hidden For example inWindows we often use an Explorer window to display afile system with a balanced view of the file structure inthe left pane and the contents of a directory in the rightHowever for large file systems this window cannot tellusers where the directory of interest is actually locatedin the hierarchy Although the scroll bars are helpful forbrowsing the directory list deciphering a large structurefrom a long list can be difficult for many users In ageographic-information system such as Mapquest userscan jump to different maps for scale-related informa-

Figure 1 Multiscale structures of the Earth (Courtesy of CERN) Figure 2 Multiscale structures of DNA (Morrison amp Morrison

1994)

32 PRESENCE VOLUME 14 NUMBER 1

tion as seen in Figure 3 but making sense of the rela-tionship between these different maps across scales maydemand considerable cognitive resources for memoryand information processing Users need help controllingthe scale levels of observation and analysis

To address these multiscale issues we propose a newtype of interaction environment the multiscale collabo-rative virtual environment (mCVE) Multiscale technol-ogy allows people to control their observation and ac-tion domains dynamically Collaboration technologybenefits people because of the division of labor and theparallel working processes Combining multiscale andcollaboration helps to better allocate individual usersrsquocognitive resources to different scales and then exploitthe resulting different perception and action capabilitiesThis multiscale collaboration technique is discussed inthe context of 3D virtual environments which can visu-alize more complicated structures mCVEs allow usersto work at different scale levels being the size of ants toobserve fine details and maintain high action accuracy

and being the size of giants to see the big picture ofstructures and have a broad interaction range By collab-orating from different scale levels in mCVEs users canbridge the scale barrier

This paper begins with the introduction of mCVEsand literature review Then it advances to design issuesrelated to the visualization of multiscale structurescross-scale information sharing and cross-scale actionAfter a discussion of the implementation of an mCVEprototype system the paper describes an experimentalstudy of the effectiveness of multiscale collaboration

2 Multiscale Collaborative VirtualEnvironments

The design of mCVEs deviates from the path setby Ivan Sutherland which is to make computer-simulated worlds real (Sutherland 1965) By going be-yond being real (Hollan amp Stornetta 1992 Stappers

Figure 3 Maps of a US city at different scales (Images provided by wwwmapquestcom)

Zhang and Furnas 33

Gaver amp Overbeeke 2000) we can expand designspace and find new ways to address old problems Thissection discusses the features of multiscale virtual envi-ronments (mVEs) the necessity of multiscale collabora-tion and the benefits of mCVEs Relevant literature isalso reviewed

21 Multiscale Virtual Environments

The absence of real-world physics in virtual envi-ronments has led to some innovative designs in virtualenvironments Teleportation for example is a usefulnavigation tool that transcends speed and time AnmVE is a design that overcomes the scale barrier andallows users to manipulate the scale of the virtual spacein their work

211 Scale in Virtual Environments In com-puter graphics scale usually refers to the rendered sizeof objects To manipulate the scale of a virtual environ-ment is to resize the virtual environment Figure 4shows images of a DNA structure at two scales Thesetwo images differ only in rendered sizes and resultingdifferent levels of detail However if the scale differenceis more dramatic image difference can go beyond geo-metric size as seen in Figure 5 where three images atsignificantly different scales show very different and im-portant characteristics the double helix in (a) the mo-lecular structure in (b) and the nucleus and the electroncloud in an atom in (c)

212 Scale in Multiscale Virtual Environ-ments This scale-dependent rendering technique dis-tinguishes mVEs from conventional virtual environ-ments Conventional virtual environments renderobjects with the same representation regardless of userinterests mVEs render objects with different representa-tions corresponding to different scale ranges In mVEsusers can easily change their observation scales and re-ceive different representations from big pictures to localdetails

Another important but less obvious characteristic ofmVEs is that users also get different interaction do-mains To change the scale of a virtual environment isto shift a userrsquos action capabilities One action can leadto different results at different scales For example auserrsquos movement could be at the nanometer level whenthe user is at the atomic scale and the same movementwould be at the micrometer level when the user movesto the molecular scale When the user observes a virtualplanet system the movement scale can be at the level ofhundreds of thousands of kilometers In mVEs userobservation and action capabilities are coordinated atthe same or comparable scale level

Users need this association in mVEs so that they canwork appropriately with what they see For examplewhen users observe an object structure at the level of 1mm their movement scale should be at a comparablescale say 1 mmsec Moving at a very different scalesay 1 kmsec would make it hard to keep a consistentview Objects of interest would be constantly speedingout of sight Conversely if the structure of interest is atthe level of 1 km moving at a scale of 1 mmsec would

Figure 4 DNA rendered at two different scales

Figure 5 Successive views of a DNA structure rendered at different

scales Perspective lines added for the figure to show where each

successive view comes from

34 PRESENCE VOLUME 14 NUMBER 1

be problematic It would demand tremendous effort tochange the view of the large structure even slightlyOther interaction parameters such as reachable distanceand manipulation accuracy should be similarly coordi-nated

Thus mVEs provide users with new interaction capa-bilities that are usually unavailable in conventional vir-tual environments In mVEs users can interact withvirtual worlds at multiple scale levels At any given scalesome things will be easy to see and manipulate whileothers will be either too large or too small to displayChanging the scale moves new things into the easilyvisible and manipulative range Coordinated observationand action capabilities help users to better interact withmultiscale structures Multiscale extends the designspace for information visualization and user interaction

22 Multiscale Collaboration

Multiscale collaboration can further facilitate userinteraction with multiscale structures In particular itcan help people with work on large structures that needmore labor on rapidly changing structures that demandparallel working and on complex structures that requiredomain knowledge beyond what an individual mayhave

221 Large Multiscale Structures Managinga large multiscale structure could be difficult for a singleuser Take a city-planning example Adding new designelements (eg paths or landmarks) may force plannersto move back and forth frequently between differentscales and check the impact of new additions on designat different levels For a large city continuously chang-ing scale could be very demanding and costly if there isjust one planner Multiscale collaboration can offersome help by placing individual planners with differentscales and having them focus on issues at their own scalelevel Planners can then work independently on prob-lems within their own scale scopes and collaborativelydeal with cross-scale issues Collaboration divides bigcross-scale problems into smaller within-scale ones thatcan be more easily handled by individuals

222 Rapidly Changing Multiscale StructuresIt could also be a challenge for a user to manage a struc-ture that changes rapidly however small that structuremay be Consider a scenario in which a small computernetwork managed by a system administrator is beinginfected by a worm virus The system administrator maynot be able to keep pace with the speed at which thevirus spreads If more system administrators are in-volved they can have individuals to check subnets atdifferent levels simultaneously and then take appropri-ate actions to stop the virus from further damaging theentire network Collaboration by supporting parallelworking improves peoplersquos response to problems atdifferent levels

223 Cross-Domain Multiscale StructuresChallenges may also come from the diversified knowl-edge and expertise required in understanding and man-aging structures spanning across the boundaries of sci-entific domains Understanding cross-domain multiscalestructures could be daunting for individuals For exam-ple in studying the strength of a new material acrossdifferent scale levels no single person would possess allthe required in-depth knowledge of chemistry materialsscience and mechanical engineering Chemists materi-als scientists and mechanical engineers may need towork together on this issue While they can focus onproblems in their own expertise domains individuallyworking together helps them see how properties at dif-ferent scale levels may affect each other and even dis-cover important issues that might be ignored withoutcross-scale collaboration Collaboration helps solvecross-scale problems by combining peoplersquos knowledgedomains from different scales

23 Multiscale Collaborative VirtualEnvironments

An mCVE is an mVE that supports collaborationIn mCVEs users can choose their own scales in the ob-servation of an object and work together on the sameobject at different scales A team of city planners forexample can have a regular planner of the normal hu-man size at street level and a ldquogiantrdquo planner at the city

Zhang and Furnas 35

level While each planner focuses on planning issues athis or her own level they can help each other on cross-scale issues The giant planner can take the street-levelplanner quickly to a distant destination and the street-level planner can help the partner place buildings orlandmarks at their exact locations Figure 6 shows twoplanners at two scales working together Their cross-scale collaboration could help make a plan that meetsthe requirements at various levels

As seen in Figure 6 each user is represented as anavatar in mCVEs with a scaled size corresponding tothe scale level of the user This avatar metaphor offerstwo benefits First avatars provide rich awareness infor-mation (eg user view locations viewing directionsbody gestures) that are critical to social interactions invirtual environments (Benford Bowers Fahlen Green-halgh amp Snowdon 1995) Second by yoking differentobservation and action parameters under the same ava-tar body avatars can help users better understand theassociation of different observation and action capabili-ties in mCVEs The eye level of a userrsquos avatar tells howfar the user can see the avatarrsquos arm length indicateshow far the user can reach the leg length shows howfast the user can move Seeing the size of a userrsquos avatarother users can tell at what scale the user is working andwhat the user can do

mCVEs could be a promising tool to help people dealwith multiscale structures but this new design paradigmraises many research issues ranging from the conceptualunderstanding of the implications of multiscale for userperception and action to the technical design of

mCVEs to the experimental evaluation of the effective-ness of mCVEs

24 Literature Review

Some research effort has been made to addressdesign issues related to user interaction with large struc-tures Furnas (1986) proposed a theoretical frameworkGeneralized Fisheye Views to manage large hierarchicalstructures by allocating more space for objects of inter-est This technique has been used to visualize varioustypes of data (Sarkar amp Brown 1992 Robertson ampMackinlay 1993 Lamping Rao amp Pirolli 1995 Raabamp Ruger 1996) Koike and Yoshihara (1993) used thefractal concept to expand the tree structure based on thedegrees of interest Cone Trees were built to present filehierarchies (Robertson Mackinlay amp Card 1991)These methods focused only on better using screenspace to deal with the size of information structures butgave little consideration to visualizing structures withdifferent representations based on user interactionscales Multiscale technology (Perlin amp Fox 1993 Bed-erson amp Hollan 1994 Lieberman 1994) providedearly inspiration for this research This technique wasusually regarded as an alternative user interface in 2DThis research extends it into 3D worlds and argues thatmultiscale technology is a powerful design that by al-lowing people to control their observation and actiondomains at different scales can augment our limitedcognitive capabilities

Scaling has been seen in some designs to support useractions in virtual environments Scaling tools were usedto increase reaching distance (Poupyrev BillinghurstWeghorst amp Ichikawa 1996 Mine Brooks amp Sequin1997 Pierce et al 1997) facilitate spatial knowledgeacquisition (Stoakley Conway amp Pausch 1995) andimprove navigation performances (Mackinlay Card ampRobertson 1990) These scaling tools usually focusedon only one interaction parameter and ignored the im-plications of the change of one parameter for all otherrelated perception and action While isolating individualinteraction parameters may work well in some situa-tions uncoordinated perception and action as discussed

Figure 6 Two city planners working at two different scales

36 PRESENCE VOLUME 14 NUMBER 1

previously could cause some problems in the interac-tion with large multiscale structures

Research literature on virtual environments support-ing collaboration is massive but consideration of cross-scale collaboration is rarely seen The most relevantproject is the CALVIN system (Leigh Johnson amp De-Fanti 1996) in which two users can collaborate on adesign project from two different perspectives In thesetwo perspectives objects were rendered with same rep-resentations and the system did not provide users withmultiple levels of abstraction at different scales Alsothe system gave users limited choices of working scalesand could not satisfy their needs for active and interac-tive control over scale across a much broader range

3 Design of mCVEs

There are some design guidelines for 3D collabo-rative virtual environments (CVEs) (Benford SnowdonColebourne OrsquoBrien amp Rodden 1997 EPFL et al1997 Tromp 1999) However they were constructedfor the design of conventional CVEs and common col-laboration actions in CVEs Applying them in designingmCVEs could be difficult because scale issues have notbeen considered in these guidelines In this research weadopt a design approach that starts from understandingthe implications of multiscale technology for user inter-actions and then derives design considerations based onthis understanding In this section efforts are made firstto analyze how multiscale may affect user interactionThen some conceptual designs are proposed to supportuser perception and action in mCVEs

31 Implications of Multiscale for UserInteraction

Multiscale could affect user interactions at differ-ent levels and in different ways Here we first discusshow multiscale can support the understanding of multi-scale structures Then we describe how it may impedeinformation sharing among users at different scales andhow it may facilitate cross-scale action

311 Understanding Multiscale StructuresAs discussed previously interacting with large structuresrequires users to obtain both detailed content and suffi-cient context information This well-known ldquocontent

contextrdquo problem is indeed a multiscale problem be-cause detailed content information and global contextinformation are usually distributed at different scale lev-els To understand complicated structures that presentdifferent features at different scales we usually breakphenomena into components and focus our attentionson issues within certain levels rather than all levels atonce (Pattee 1973 Ahl amp Allen 1996) and thenchoose appropriate tools (eg microscopes telescopes)for observation and analysis at different scales

In virtual environments multiscale technology couldhelp address this ldquofocus contextrdquo problem by allowingusers to control what they want to see and what they cansee from multiscale structures and then to manipulate theamount of context and content information displayed onthe screen Under different interaction scales objects canbe rendered at different sizes tiny objects can be madelarger for more detailed observation and huge objects canbe made smaller for a better big picture The same struc-ture can have different appearances at different scales in-forming users with the rich features of the structure asseen in Figure 5 The understanding of a complicatedstructure could be improved

312 Cross-Scale Information Sharing Col-laboration can facilitate peoplersquos work by improvingcommunications andor cooperation on shared artifacts(Dix 1994) Multiscale collaboration focuses more onartifacts than communications In such artifact-centeredcollaboration users often need to refer to shared arti-facts (Dix) Conventional CVEs are usually ldquoWhat-You-See-Is-What-I-Seerdquo (WYSIWIS) or ldquorelaxed WYSIWISrdquo(Stefik Bobrow Foster Lanning amp Tatar 1987) soknowing what others are referring to is fairly easymCVEs are not like this however Users may have verydifferent views at different scales and see totally differ-ent objects and structures Figure 7 shows the view of acity planner who oversees the whole city (a) and theview of another planner who checks buildings at thestreet level (b) These two views which are from very

Zhang and Furnas 37

different perspectives and show different levels of ab-straction of the city may make it difficult for two plan-ners to understand how their work can be related Aplanner can refer to a particular object in her view butthe other may not see it at all Multiscale technologymay hurt mutual understanding of objects in collabora-tion because of scale-related views

313 Cross-Scale Action Multiscale can alsobring some benefits to collaboration One of them isthat users can leverage their different action capabilitiesand different scopes of influence of their actions Takethe example of the two city planners again When thestreet-level planner needs to go to a distant place thecity-level planner with a great action domain can movethe partner easily to the destination and reduce the tasktime Or when the city-level planner wants to move toan exact location the street-level planner can move thepartnerrsquos avatar accurately because of high manipulationaccuracy It is mentioned previously that a similar tech-nique can be used in object manipulation Thus multi-scale collaboration can improve navigation and manipu-lation activities by allowing users to take advantage ofeach otherrsquos different action capabilities

32 Design Considerations

Since multiscale technology could affect user in-teraction both positively and negatively the design ofmCVEs focuses on maximizing the advantages of multi-scale technology and avoiding the disadvantages Thisprinciple leads to the following design considerations

321 Support for Better Understanding ofMultiscale Structures Multiscale structures shouldbe modeled in such a way that scale-dependent charac-teristics can be observed at different scale levels In thereal world multiscale structures are ubiquitous as seenin Figures 1 and 2 To observe these multiscale struc-tures we need only appropriate tools In 3D virtualworlds however what users can see is predesignedThus to visualize multiscale structures objects have tobe modeled and rendered in a multiscale way

322 Support for Cross-Scale InformationSharing Users should be able to share artifacts ofcommon interest with others across scale Tools such asmultiple views of the world (Gaver Sellen Heath ampLuff 1993) cannot help too much Such techniquesrequire some common objects in different views as ref-erences but usersrsquo very subjective views in mCVEs maymake it difficult to find such references It is a challengeto build mutual understanding of working contexts withsubjective views (Snowdon amp Jaa-Aro 1997) Usersneed help in understanding how their subjective viewsand objects in them are related to each other

323 Support for Cross-Scale Action Usersshould be allowed to get involved in othersrsquo work acrossscale so that they can leverage their different interactioncapabilities in collaboration They should be able to di-rectly manipulate objects that others are working onand move the avatars of other users Of course this in-tervention should be based on mutual agreements toavoid any unwanted consequences In the city-planningexample the city-level planner should not change thingsthe street-level planner is working on without consentOtherwise the street-level planner may get confusedand not understand what has happened

33 Conceptual Design

Based on these considerations some multiscaletools were designed Here discussions focus on designsthat are fundamentally important to mCVEs includingscale-based semantic representations that present scale-related characteristics of multiscale structures dynamic

Figure 7 Two views of a city at two scale levels

38 PRESENCE VOLUME 14 NUMBER 1

view transition that facilitates cross-scale informationsharing and direct avatar manipulation that supportscross-scale action

331 Scale-Based Semantic Representationsfor Multiscale Structures This design renders amultiscale structure with different representations atdifferent scales Figure 5 presents the visual results of asubstance under this technique Initially the substanceappears as a double helix in (a) The user changes thescale to increase the structure size and to see more de-tails in (b) Continuing the scaling the user finds a dif-ferent structure in an atom in (c) The user is informedof semantic characteristics of the substance at differentscales

This scale-based representation technique has oneproblem The change between two very different repre-sentations may confuse users For example the suddenappearance of the structure in Figure 5(c) may distractusers from their primary tasks To address this issue weadopted a fading tool (Bederson amp Hollan 1994Lieberman 1994) to smooth the transition betweendifferent representations Figure 8 shows a transitionprocess from a spherelike molecule from Figure 8(a) toa set of atoms inside it in Figure 8(d) Instead of jump-ing abruptly between different representations the dis-appearance of the molecule and the appearance of atomsare gradual The transition between two representationsis more easily comprehensible Cognition required inunderstanding the change of visual image can be kept atthe level of perceptual processing so that users can freetheir cognitive processing capacities for more compli-cated tasks (Robertson Card amp Mackinlay 1991)

Scale-based semantic representations can be seen asone member of the family of level of detail (LOD) tech-

niques which visualize the same structure with differentgeometric objects according to a particular interactionparameter but with a very different purpose Whilemost LOD techniques such as distance LOD primarilyconcern computation efficiency (Puppo amp Scopigno1997) scale-based semantic representation gives moreconsideration to the userrsquos interaction needs CommonLOD techniques render individual objects with differenttextures or geometries but scale-based semantic repre-sentation visualizes structures with different semanticabstractions LOD techniques can be integrated intoscale-based semantic representations to improve bothuser interaction and machine performance For examplein a city-planning project a city can have different struc-tures at the level of city district and street to informplanners semantically and distance LOD can be appliedin rendering individual objects in each structure to im-prove 3D computation

332 Dynamic View to Support Cross-ScaleInformation Sharing Design here addresses issuesconcerning how to help users at different scales to buildcommon understandings of their divergent views Onedesign choice is to use a dynamic view to narrow downthe view difference We used animation to show thetransition between two views Animation is generatedby interpolating the view positions view orientationsand view scales of two views For example if two usersrsquoviews at two different scales look like Figures 7(a) and7(b) an animation can be generated by adding interme-diate image frames between Figure 7(a) and 7(b) Theanimation can tell how one view can be transformed toanother and how objects in two views are related toeach other

333 Direct Avatar Manipulation to SupportCross-Scale Action To help users get involved inothersrsquo work so that they can take advantage of theirdifferent action capabilities we adopted a design choicethat allows direct manipulation on other usersrsquo avatarsWith such a tool planners in the above city-planningexample can easily cooperate on tasks that require largeaction domain and high action accuracy Of course thisdirect interference should be regulated The user who

Figure 8 Successive fading views in semantic representations

Zhang and Furnas 39

owns the avatar is always aware of such action and hascontrol over whether this action is allowed

The implication of such a direct manipulation of ava-tar body for user interface design deserves more atten-tion Under this design an avatar can ldquoaffordrdquo manipu-lation activities This is quite different from thetraditional role of an avatar merely as user embodimentin conventional CVEs The reason that no direct manip-ulation of an avatar is allowed in conventional CVEsmight be due to the influence of real life where the au-tonomy of human bodies is well respected To betterfacilitate peoplersquos work in virtual environments somereal-life constraints could be relaxed or even abandonedVirtual environments should not be treated as the repli-cation of the real world just as new technologies shouldnot be interpreted by old metaphors Direct manipula-tion of a userrsquos avatar by others is an effort to breakdown the metaphor of reality

4 System Implementation

A desktop mCVE prototype system consisting ofover 20000 lines of Java code was implemented Dis-cussions in this section focus on the choice of toolkits insoftware implementation general system architectureand the implementation of some multiscale tools

41 Choice of Toolkits

The design of virtual environments that supportcollaboration needs to consider issues in three aspects3D graphics rendering network communication man-agement and user interactions (Singhal amp Zyda 1999)It is not our interest to address issues related to low-level graphics rendering or network communicationsOur goal here is to support user interactions with multi-scale structures and with other users at different scalesTo achieve this interaction-oriented design goal weimplemented the mCVE prototype with two toolkitsJava 3D and Java Shared Data Toolkit (JSDT)

This hybrid approach allows a design focus on high-level interaction tools because Java 3D can handlegraphics rendering and JSDT can handle network com-

munications issues With the scene-graph managementof Java 3D we can construct virtual environments andstructures at the level of geometric objects rather thanat the level of vertices and polygons Java 3D allows theextension of new rendering behaviors which is not wellsupported by other CVE toolkits such as DIVE (Carls-son amp Hagsand 1993) and SPLINE (Waters et al1997) This flexibility is critical to the implementationof such tools as scale-based semantic representations anddynamic view transition Furthermore the easy integra-tion of other Java APIs such as Java Swing into Java3D can simplify user interface design JSDT is ideal forthe research because it provides flexible data deliveryservices in support of synchronous collaboration on theInternet JSDT supports basic abstraction of networksessions multicast message communications and con-currency control Thus the design can focus on opti-mizing the management of distributed 3D scenes andcross-scale communications One drawback of this hy-brid approach is the lack of those CVE services seen inSPLINE (Waters et al) Collaboration services have tobe built from scratch

42 System Layer and Architecture

The system layer of the prototype is shown in Fig-ure 9 The implementation focused on the two layersinside the dashed rectangle On top of Java 3D andJSDT are multiscale graphics services and collaboration

Figure 9 System layer of the mCVE prototype

40 PRESENCE VOLUME 14 NUMBER 1

services and above them are multiscale collaborationtools

Multiscale graphics services handle object-renderingissues and update user views based on their view posi-tions and interaction scales Collaboration services de-liver messages among distributed users and manage theconsistency of shared virtual environments Multiscalecollaboration services deal with such collaboration issuesas cross-scale information sharing and cross-scale avatarmanipulation These services lay the foundation for multi-scale tools

43 Implementation of Multiscale Tools

A set of multiscale collaboration tools were imple-mented and integrated into the prototype system Thissection introduces such tools as scaling control whichdistinguishes mCVEs from other CVEs scale-based se-mantic representations which visualize information in amultiscale way cross-scale dynamic view transitionwhich addresses the cross-scale information problemuniquely seen in multiscale collaboration and the man-agement of avatar position in direct avatar manipulationwhich illustrates cross-scale data coordination

431 Scene Graph of the Prototype Systemand Scaling Control The scene graph of the systemis shown in Figure 10 Each Java 3D scene graph has aVirtual Universe object as the root A Locale object at-taches scene graphs to the Virtual Universe objectBranchGroup (BG) nodes collect such objects as geom-etry behavior light sound and so forth A Transform-Group (TG) node defines the coordinates of objectsunder it

The system has two subscene graphs the object scenegraph which is the left child of the Locale object andthe view scene graph which is the right child The ob-ject scene graph collects all objects and behavior nodesThe view scene graph defines the viewing behaviors

To change a userrsquos interaction scale is to change thevalues of the userrsquos interaction parameters relative to thesize of the virtual world Thus scale change can be im-plemented in two ways (1) scaling the virtual world upand down and keeping all interaction parameters un-

touched or (2) changing all interaction parameters andkeeping the virtual world untouched Mathematicallythey are equivalent We adopted the first approach be-cause scaling the virtual world up and down can bedone by modifying just one parameter the scaling fac-tor Si of the top TG in the object scene graph This ismuch simpler than the second approach which involvesthe modification of complicated view parameters

432 Scale-Based Semantic RepresentationsIn the implementation of scale-based semantic represen-tations we adopted a design that uses a delegate objectto wrap all representations and that delivers appropriaterepresentations based on the value of the controllingvariable (Fox 1998) A scale-based semantic representa-tion node is extended from the Java 3D Switch objectEach node has a set of child representations and an or-dered array that defines the scale range for each repre-sentation (Figure 11)

433 Animating View Transition in Collabo-rative View Sharing The key issue in making a view-transition animation is to interpolate two views InmCVEs a view can be uniquely determined by its viewposition P its view orientation O and its view scale Sand be written as V(P O S)

Given two views V0 V(P0 O 0 S0) and V1 V(P1O 1 S1) to implement a view-transition animation is to

Figure 10 Scene graph of the prototype system

Zhang and Furnas 41

choose a path that links V0 and V1 and to set the trajec-tory of viewpoint movement view direction along thepath and observation scale along the path View-pathand view-orientation interpolation techniques are ma-ture (Parent 2001 Watt amp Policarpo 2001) so thisresearch focused only on the interpolation of view scale

A scale-interpolation function could be linear loga-rithmic or in other forms We chose a logarithmic func-tion because it gives users a constant relative rate ofchange in their views Interpolating the view scale loga-rithmically can give users a familiar experience

Assume the animation includes n frames from V0 toV1 The view scale of the ith frame (0 i n) in a log-arithmic interpolation function can be written as

Si Si S0 (1)

where

S S1

S01n

(2)

With corresponding P i and O i from interpolationfunctions for view position and view orientation theview of the ith frame Vi V(P i O i Si) can be ob-tained Figure 12 shows two views seen in Figure 7 asFrame 0 and Frame n and other interpolated viewframes (Due to space limitations we have shown onlyfour frames used for the animation) Showing theseframes successively with a reasonable frame rate can pro-duce an animation linking these two divergent views

434 Cross-Scale Data Consistency and Ava-tar Position in Direct Avatar Manipulation Al-though changing the scaling factor of the top TG in theobject scene graph simplifies the scaling operation such

an approach poses a challenge in cross-scale data consis-tency In CVEs all users usually need to see the sameobject appearing at the same spatial location In conven-tional CVEs this can be easily achieved because an ob-jectrsquos coordinates in different usersrsquo worlds are often thesame In mCVEs however when users are at differentscales their virtual worlds are rescaled according to thescaling factors of the top TG in their own object scenegraphs The same object will have different local coordi-nates in different usersrsquo worlds When the object ismoved by a user the objectrsquos local coordinates in thisuserrsquos world cannot be used to update the objectrsquos newposition in all other usersrsquo worlds

In the implementation we used normalized coordi-nates which are calculated as the ratio of the object lo-cal coordinates in the userrsquos world to the userrsquos scaleLet Pl(xl yl zl) and Sl denote the local coordinates andthe userrsquos scale respectively The normalized coordi-nates Pn can be written as

Pn PlSl (3)

By doing so the same object will always have the samePn This Pn is used to define the coordinates of the ob-ject in every userrsquos object scene graph Modifying thespatial location of an object in a userrsquos world leads to anew Pn All users can use this Pn to update the coordi-nates of the object in their object scene graphs There-fore Pn enforces data consistency across scale Becausethe scaling factor is on the top of the scene graph theobjectrsquos final location in a userrsquos world is Pl the productof Pn and this userrsquos scale Sl With different Sl users willbe able to see the world rendered at different scales

An avatarrsquos position can be managed similarly butwith a small difference An avatar object differs fromother ordinary objects in a way that in its host userrsquos

Figure 11 Scale-based semantic representation node

Figure 12 Frames of a dynamic view-transition animation

42 PRESENCE VOLUME 14 NUMBER 1

world the coordinates of the avatar are affiliated withthe view scene graph rather than the object scenegraph so that the avatar always moves with the view Inother usersrsquo worlds however the avatar object is underthe object scene graphs Figure 13(a) is the scene graphof the world of a user say A who interacts with the vir-tual environment at a location Pa and with a scale SaFigure 13(b) is the scene graph of the world of anotheruser say B User Arsquos avatar is in User Brsquos object graphBecause Pa is not governed by Sa the normalized coor-dinates of the avatar in User Brsquos object graph should bewritten as

PnPaSa (4)

The body size of User Arsquos avatar changes with thescale of User A so a scale parameter is also needed todefine the size of User Arsquos avatar in User Brsquos objectscene graph so that User B can be aware of User Arsquosscale This scale should also be normalized Let Sn de-note the normalized scale where Sn is simply the recip-rocal of Sa

Sn 1Sa (5)

This is because the avatar body changes inversely pro-portional to the scale factor Sa (shrinking the world isequivalent to growing the avatar body and growing theworld is equivalent to shrinking the avatar body)

The implementation of other cross-scale tools includ-ing direct avatar manipulation also uses Pn and Sn todefine the coordinates and size of an avatar entity Thisapproach guarantees the consistency of the avatar posi-tion and size in all usersrsquo views at different scales

5 Experimental Evaluation

We evaluated multiscale collaboration in subjecttests An experiment was designed to assess the benefitsusers can get from an mCVE in accomplishing tasks re-quiring cross-scale coordination

51 Experiment Design and Procedure

Subject tasks in the experiment involved searchingfor a ldquobombrdquo on a square ground plane (2000 2000m2) with a distinctly shaped building in each corner(square hexagon octagon and circle as seen fromabove) Each had a height of 12 m and a base of about6400 m2 On the ground behind each building was aunit cube (1 m3) containing a unique text name andsmiley face (about 05 05 m2) The bomb was insideone of these four cubes In the test subjects were placedin the middle of the square plane They had a defaulteye level of 168 m and a default moving step of 1 m inthe virtual environment The shape of the building thatthe bomb was nearby was known in advance and sub-jects needed to find that building locate the bomb boxand key in the name of the box to defuse the bomb

A 2 2 2 factorial design that has six treatmentstwo noncollaboration and four collaboration wasadopted (Table 1) For the two noncollaboration treat-ments VE is just a conventional 3D virtual environ-ment and M-VE is a VE enhanced by multiscale toolsspecifically it allows users to change their interactionscales eye level and speed Among four collaborationtreatments CVE a conventional collaborative virtualenvironment is the only one without multiscale toolsThe other three treatments all equipped with multiscale

Figure 13 Two scene graphs for the same avatar

Table 1 Six Treatments in a 2 2 2 Design

Noncollaboration Collaboration

Nonmultiscale VE CVEMultiscale M-VE NR

GUIDEMOVE

Zhang and Furnas 43

tools differ in the assignment of subject task roles (be-ing a giant or a normal person) and in the way subjectsaffect each otherrsquos work across scales In one treatmentthe roles of subjects are not predefined and subjects canchoose their own interaction scales as desired This envi-ronment is labeled as NR (No Role) The other twomCVEs both assign one subject to be a giant and theother a normal person so that subjects can change theirinteraction scales only within a limited range In onesuch environment the giant is permitted to move thepartner directly and this treatment is labeled as MOVEAnother condition allows the giant to guide the move-ment of the partner only verbally and is denoted asGUIDE

Recruited through email 24 students paired in 12groups participated in the experiment In noncollabora-tion treatments they worked on their own and in col-laboration treatments they communicated through anaudio channel The performances were measured bytask-completion time

52 Results

An ANOVA analysis of data from four treatments(VE M-VE CVE and MOVE) shows main effects ofboth collaboration (F170 1298 p 0001) andmultiscale (F170 7090 p 00001) Interaction isnot significant (F170 187 p 176) as seen in Fig-ure 14 Subjects performed best in MOVE where theycould take full advantage of multiscale and collabora-tion and they did worst in VE where there was no helpat all

Subjects used different strategies in the different con-ditions Without multiscale tools subjects had to goaround and count the number of sides of all four build-ings to find the target a very time-consuming processWith multiscale subjects could increase their sizes to seebuilding shapes from above and approach the bombquickly so the time required was be reduced signifi-cantly

The performance difference among the three mCVEsalso indicates the importance of different collaborationsupports In the NR treatment subjects had to assumetheir roles via an expensive negotiation process In the

GUIDE treatment the giant subject who could see thebuilding shape must coordinate with the partner aboutnavigation verbally through a costly grounding process(Clark amp Brennan 1991) In MOVE however the all-seeing giant could actually move the partner to the des-tination quickly and bypassed grounding processes

6 Discussion

In this paper we have introduced a novel interac-tive environment multiscale collaborative virtual envi-ronment which gives users the freedom to control theirinteraction scales in collaboration Multiscale collabora-tion can help better allocate scarce scale resources andcombine different expertise and action capabilities ofindividual users in their work with large structures Wehave also discussed the implications of multiscale tech-nology for user interaction and the design and develop-ment of a Java-based mCVE system Our subject testshave shown that multiscale collaboration can facilitatepeoplersquos work that requires cross-scale information andcollaboration

Research reported in this paper has largely focused ongeneral interaction issues in mCVEs Designs have beenlimited to generic tools to support multiscale collabora-tion and experimental study has focused on generaltasks We expect that multiscale collaboration tools canbe used to support peoplersquos specific work in the realworld To achieve this goal the design of multiscale

Figure 14 Time (in seconds) for the six treatments

44 PRESENCE VOLUME 14 NUMBER 1

tools should support domain-specific tasks One of ourongoing research projects focuses on studying how multi-scale collaboration can support research in materials sci-ence and engineering We are investigating the needs ofmaterials scientists for multiscale collaboration and willdesign and implement tools that support these needsThe effectiveness and usability of these tools will also beevaluated when they are ready

At the same time we are interested in behavior issuesin mCVEs Some important research questions concern-ing user behavior in mCVEs have not been studiedthoroughly It is still unclear when and how users canbenefit from collaborative object manipulation acrossscale whether the designed dynamic view-transitiontechnique can indeed improve cross-scale informationsharing what mechanism is needed to help users bettermanage cross-scale collaboration overheads and soforth In addition it has been found that multiscale alsobrings some new challenges in social interactions(Zhang amp Furnas 2002) and efforts are needed tostudy what can be done to address these interaction is-sues Research in this direction will deepen our under-standing of mCVEs and help to design better tools tosupport multiscale collaboration

Acknowledgments

This research was funded in part by Microsoft Research

References

Ahl V amp Allen T (1996) Hierarchy theory A vision vocab-ulary and epistemology New York Columbia UniversityPress

Alexander C Ishikawa S amp Silverstein M (1977) A pat-tern language Oxford UK Oxford University Press

Bederson B amp Hollan J (1994) Pad A zoominggraphical interface for exploring alternate interface physicsProceedings of ACM UIST rsquo94 17ndash26

Benford S Bowers J Fahlen L Greenhalgh C amp Snow-don D (1995) User embodiment in collaborative virtualenvironments Proceedings of ACM CHI rsquo95 242ndash249

Benford S Snowdon D Colebourne A OrsquoBrien J ampRodden T (1997) Informing the design of collaborativevirtual environments Proceedings of ACM GROUP rsquo97 71ndash80

Carlsson C amp Hagsand O (1993) DIVEmdashA platform formulti-user virtual environments Computer and Graphics17(6) 663ndash669

Clark H amp Brennan S (1991) Grounding in communica-tion In L B Resnick J Levine amp S D Behrend (Eds)Perspectives on socially shared cognition (pp 127ndash149)Washington DC American Psychological Association

Dix A J (1994) Computer-supported cooperative workmdashAframework In D Rosenburg amp C Hutchison (Eds) De-sign Issues in CSCW (pp 23ndash37) Berlin Springer-Verlag

EPFL Geneva IIS TNO Lancaster UCL et al (1997)Guidelines for building CVE applications COVEN ProjectRetrieved November 21 2002 from httpwwwcsuclacukstaffmslatervrCovenPublicDeliverablesdel26PDF

Fox D (1998) Tabula rasa A multi-scale user interface sys-tem Unpublished doctoral dissertation New York Univer-sity

Furnas G (1986) Generalized fisheye views Proceedings ofACM CHI rsquo86 16ndash23

Gaver W Sellen A Heath C amp Luff P (1993) One isnot enough Multiple views in a media space Proceedings ofACM INTERCHI rsquo93 335ndash341

Hollan J amp Stornetta S (1992) Beyond being there Pro-ceedings of ACM CHI rsquo92 119ndash125

Koike H amp Yoshihara H (1993) Fractal approaches forvisualizing huge hierarchies Proceedings of IEEE VL rsquo9355ndash60

Lamping J Rao R amp Pirolli P (1995) A focus contexttechnique based on hyperbolic geometry for visualizinglarge hierarchies Proceedings of ACM CHI rsquo95 401ndash408

Leigh J Johnson A amp DeFanti T (1996) CALVIN Animmersimedia design environment utilizing heterogeneousperspectives Proceedings of IEEE International Conferenceon Multimedia Computing and Systems 20ndash23

Lieberman H (1994) Powers of ten thousand Navigating inlarge information spaces Proceedings of ACM UIST rsquo9415ndash16

Mackinlay J Card S amp Robertson G (1990) Rapid con-trolled movement through a virtual 3D workspace Proceed-ings of ACM SIGGRAPH rsquo90 171ndash176

Mine M Brooks F amp Sequin C (1997) Moving objects

Zhang and Furnas 45

in space Exploiting proprioception in virtual-environmentinteraction Proceedings of ACM SIGGRAPH rsquo97 19ndash26

Morrison P amp Morrison P (1994) Powers of ten About therelative size of things in the universe and the effect of addinganother zero San Francisco W H Freeman

Parent R (2001) Computer animation Algorithms and tech-niques San Francisco Morgan-Kaufmann

Pattee H (1973) Hierarchy theory The challenge of complexsystems New York George Braziller

Perlin K amp Fox D (1993) Pad An alternative approach tothe computer interface Proceedings of ACM SIGGRAPHrsquo93 57ndash64

Pierce J Forsberg A Conway M Hong S Zeleznik Ramp Mine M (1997) Image plane interaction techniques in3D immersive environments Proceedings of the 1997 Sympo-sium on Interactive 3D Graphics 39ndash43

Poupyrev I Billinghurst M Weghorst S amp Ichikawa T(1996) The go-go interaction technique Non-linear map-ping for direct manipulation in VR Proceedings of ACMUIST rsquo96 79ndash80

Puppo E amp Scopigno R (1997) Simplification LOD multi-resolution Principles and applications Proceedings ofEUROGRAPHICS 16 (3)

Raab A amp Ruger M (1996) 3D-Zoom Interactive visual-ization of structures and relations in complex graphics InB Girod H Nieman amp H Seidel (Eds) 3D image analy-sis and synthesis (pp 125ndash132) Sankt Augustin GermanyInfix-Verlag

Robbins M (2001) Modeling and simulation of the failureand toughness of polymersubstrate bonds The second An-nual Multiscale ModelingmdashBridging the Gap Between Atom-istic and Meso Scales

Robertson G Card S amp Mackinlay J (1991) Informationvisualization using 3D interactive animation Proceedings ofACM CHI rsquo91 461ndash462

Robertson G amp Mackinlay J (1993) The document lensProceedings of ACM UIST rsquo93 101ndash110

Robertson G Mackinlay J amp Card S (1991) Cone treesAnimated 3D visualizations of hierarchical informationCommunications of the ACM 34 (2) p 89ndash194

Sarkar M amp Brown M H (1992) Graphical fisheye viewsof graphs Proceedings of ACM CHI rsquo92 83ndash91

Singhal S amp Zyda M (1999) Networked virtual environ-ments Design and implementation New York ACM Press

Snowdon D amp Jaa-Aro K (1997) A subjective virtual envi-ronment for collaborative information visualization VirtualReality Universe rsquo97 Retrieved May 23 2003 fromhttpwwwcyberedgecomvru_paperssnowdenhtm

Stappers P Gaver W amp Overbeeke C (2000) Beyond thelimits of real-time realism 2000 Chapter to appear in L JHettinger amp M W Haas (Eds) Psychological issues in thedesign and use of virtual and adaptive environment Mah-wah NJ Erlbaum

Stefik M Bobrow D Foster G Lanning S amp Tatar D(1987) WYSIWIS revised Early experiences with multiuserinterfaces ACM Transactions on Information Systems(TOIS) 5(2) 147ndash167

Stoakley R Conway M amp Pausch R (1995) Virtual real-ity on a WIM Interactive worlds in miniature Proceedingsof ACM CHI rsquo95 265ndash272

Sutherland I (1965) The ultimate display Proceedings ofIFIP Congress 2

Tromp J G (1999) Usability design for CVEs COVENProject Retrieved November 21 2002 from httpcovenlancsacuk4deliverablesdel29pdf

Waters R Anderson D Barrus J Brogan D Casey MMcKeown S et al (1997) Diamond park and SPLINESocial virtual reality with 3D animation spoken interactionand runtime extendibility Presence Teleoperators and Vir-tual Environments 6(4) 461ndash481

Watt A amp Policarpo F (2001) 3D games Animation andadvanced real-time rendering New York Addison-Wesley

Zhang X amp Furnas G (2002) Social interactions in multi-scale CVEs Proceedings of ACM CVE 2002 31ndash38

46 PRESENCE VOLUME 14 NUMBER 1

may change the design coherence at the district level)To complete such multiscale tasks we need tools tocontrol the scale level of observation and analysis so thatwe can see structures at different levels and understandtheir relationships This is a challenge given that thenormal interaction scale range of human beings is onlyabout five orders of magnitude from millimeters (103

m) to hundreds of meters (102 m) The size scale rangeof matter in the real world is much larger spanningabout forty-two orders of magnitude from 1016 m to1026 m (Morrison amp Morrison 1994)

We use many different tools in real life to help uscross scale boundaries For example we have micro-scope pictures satellite pictures and scaled model struc-tures for the observation and analysis of objects or struc-tures beyond our normal interaction scale range (egDNA structures architectural structures planet sys-tems) However these artifacts give us only static infor-mation at a particular scale level It is still difficult toreceive information from other scale levels Even whenpictures or model structures at different scales are avail-able interpreting how these pictures or models are re-lated is still a challenge Computer tools for modelingsuffer from similar problems because of the lack of sup-port for multiscale interaction For example Cerius2 avery powerful tool to simulate small and ideal structures(eg unit cells of organic thin films) in materials sci-ence can hardly go beyond the molecular level forlarger structures (eg a structure with 100 100

100 unit cells) that are closer to real-world situations Inarchitectural design complex large-scale projects suchas airports have many thousands of interconnected de-signs at different scales However existing computer-

aided tools are very limited in support of the visualiza-tion organization and navigation of these designs(A Summerfield personal communication December12 2003)

Such a challenge in handling multiscale structures alsoexists when people work with information structures invirtual worlds Information structures such as file sys-tems digital libraries and the World Wide Web are rap-idly becoming larger and larger These structures caneasily have more than tens of thousands of componentsoverwhelming our memories and information-processingcapabilities We usually deal with large structures in ahierarchical way either explicitly or implicitly For ex-ample file systems are organized as explicit hierarchiesThe semantic structure that bonds the maps and relatedgeographic information in Figure 3 uses hierarchy im-plicitly

Hierarchies help us focus on issues within an appro-priate scale level but in limiting our attention to a cer-tain scale range we will lose some important informationof large structures Either detailed content or globalcontext information has to be hidden For example inWindows we often use an Explorer window to display afile system with a balanced view of the file structure inthe left pane and the contents of a directory in the rightHowever for large file systems this window cannot tellusers where the directory of interest is actually locatedin the hierarchy Although the scroll bars are helpful forbrowsing the directory list deciphering a large structurefrom a long list can be difficult for many users In ageographic-information system such as Mapquest userscan jump to different maps for scale-related informa-

Figure 1 Multiscale structures of the Earth (Courtesy of CERN) Figure 2 Multiscale structures of DNA (Morrison amp Morrison

1994)

32 PRESENCE VOLUME 14 NUMBER 1

tion as seen in Figure 3 but making sense of the rela-tionship between these different maps across scales maydemand considerable cognitive resources for memoryand information processing Users need help controllingthe scale levels of observation and analysis

To address these multiscale issues we propose a newtype of interaction environment the multiscale collabo-rative virtual environment (mCVE) Multiscale technol-ogy allows people to control their observation and ac-tion domains dynamically Collaboration technologybenefits people because of the division of labor and theparallel working processes Combining multiscale andcollaboration helps to better allocate individual usersrsquocognitive resources to different scales and then exploitthe resulting different perception and action capabilitiesThis multiscale collaboration technique is discussed inthe context of 3D virtual environments which can visu-alize more complicated structures mCVEs allow usersto work at different scale levels being the size of ants toobserve fine details and maintain high action accuracy

and being the size of giants to see the big picture ofstructures and have a broad interaction range By collab-orating from different scale levels in mCVEs users canbridge the scale barrier

This paper begins with the introduction of mCVEsand literature review Then it advances to design issuesrelated to the visualization of multiscale structurescross-scale information sharing and cross-scale actionAfter a discussion of the implementation of an mCVEprototype system the paper describes an experimentalstudy of the effectiveness of multiscale collaboration

2 Multiscale Collaborative VirtualEnvironments

The design of mCVEs deviates from the path setby Ivan Sutherland which is to make computer-simulated worlds real (Sutherland 1965) By going be-yond being real (Hollan amp Stornetta 1992 Stappers

Figure 3 Maps of a US city at different scales (Images provided by wwwmapquestcom)

Zhang and Furnas 33

Gaver amp Overbeeke 2000) we can expand designspace and find new ways to address old problems Thissection discusses the features of multiscale virtual envi-ronments (mVEs) the necessity of multiscale collabora-tion and the benefits of mCVEs Relevant literature isalso reviewed

21 Multiscale Virtual Environments

The absence of real-world physics in virtual envi-ronments has led to some innovative designs in virtualenvironments Teleportation for example is a usefulnavigation tool that transcends speed and time AnmVE is a design that overcomes the scale barrier andallows users to manipulate the scale of the virtual spacein their work

211 Scale in Virtual Environments In com-puter graphics scale usually refers to the rendered sizeof objects To manipulate the scale of a virtual environ-ment is to resize the virtual environment Figure 4shows images of a DNA structure at two scales Thesetwo images differ only in rendered sizes and resultingdifferent levels of detail However if the scale differenceis more dramatic image difference can go beyond geo-metric size as seen in Figure 5 where three images atsignificantly different scales show very different and im-portant characteristics the double helix in (a) the mo-lecular structure in (b) and the nucleus and the electroncloud in an atom in (c)

212 Scale in Multiscale Virtual Environ-ments This scale-dependent rendering technique dis-tinguishes mVEs from conventional virtual environ-ments Conventional virtual environments renderobjects with the same representation regardless of userinterests mVEs render objects with different representa-tions corresponding to different scale ranges In mVEsusers can easily change their observation scales and re-ceive different representations from big pictures to localdetails

Another important but less obvious characteristic ofmVEs is that users also get different interaction do-mains To change the scale of a virtual environment isto shift a userrsquos action capabilities One action can leadto different results at different scales For example auserrsquos movement could be at the nanometer level whenthe user is at the atomic scale and the same movementwould be at the micrometer level when the user movesto the molecular scale When the user observes a virtualplanet system the movement scale can be at the level ofhundreds of thousands of kilometers In mVEs userobservation and action capabilities are coordinated atthe same or comparable scale level

Users need this association in mVEs so that they canwork appropriately with what they see For examplewhen users observe an object structure at the level of 1mm their movement scale should be at a comparablescale say 1 mmsec Moving at a very different scalesay 1 kmsec would make it hard to keep a consistentview Objects of interest would be constantly speedingout of sight Conversely if the structure of interest is atthe level of 1 km moving at a scale of 1 mmsec would

Figure 4 DNA rendered at two different scales

Figure 5 Successive views of a DNA structure rendered at different

scales Perspective lines added for the figure to show where each

successive view comes from

34 PRESENCE VOLUME 14 NUMBER 1

be problematic It would demand tremendous effort tochange the view of the large structure even slightlyOther interaction parameters such as reachable distanceand manipulation accuracy should be similarly coordi-nated

Thus mVEs provide users with new interaction capa-bilities that are usually unavailable in conventional vir-tual environments In mVEs users can interact withvirtual worlds at multiple scale levels At any given scalesome things will be easy to see and manipulate whileothers will be either too large or too small to displayChanging the scale moves new things into the easilyvisible and manipulative range Coordinated observationand action capabilities help users to better interact withmultiscale structures Multiscale extends the designspace for information visualization and user interaction

22 Multiscale Collaboration

Multiscale collaboration can further facilitate userinteraction with multiscale structures In particular itcan help people with work on large structures that needmore labor on rapidly changing structures that demandparallel working and on complex structures that requiredomain knowledge beyond what an individual mayhave

221 Large Multiscale Structures Managinga large multiscale structure could be difficult for a singleuser Take a city-planning example Adding new designelements (eg paths or landmarks) may force plannersto move back and forth frequently between differentscales and check the impact of new additions on designat different levels For a large city continuously chang-ing scale could be very demanding and costly if there isjust one planner Multiscale collaboration can offersome help by placing individual planners with differentscales and having them focus on issues at their own scalelevel Planners can then work independently on prob-lems within their own scale scopes and collaborativelydeal with cross-scale issues Collaboration divides bigcross-scale problems into smaller within-scale ones thatcan be more easily handled by individuals

222 Rapidly Changing Multiscale StructuresIt could also be a challenge for a user to manage a struc-ture that changes rapidly however small that structuremay be Consider a scenario in which a small computernetwork managed by a system administrator is beinginfected by a worm virus The system administrator maynot be able to keep pace with the speed at which thevirus spreads If more system administrators are in-volved they can have individuals to check subnets atdifferent levels simultaneously and then take appropri-ate actions to stop the virus from further damaging theentire network Collaboration by supporting parallelworking improves peoplersquos response to problems atdifferent levels

223 Cross-Domain Multiscale StructuresChallenges may also come from the diversified knowl-edge and expertise required in understanding and man-aging structures spanning across the boundaries of sci-entific domains Understanding cross-domain multiscalestructures could be daunting for individuals For exam-ple in studying the strength of a new material acrossdifferent scale levels no single person would possess allthe required in-depth knowledge of chemistry materialsscience and mechanical engineering Chemists materi-als scientists and mechanical engineers may need towork together on this issue While they can focus onproblems in their own expertise domains individuallyworking together helps them see how properties at dif-ferent scale levels may affect each other and even dis-cover important issues that might be ignored withoutcross-scale collaboration Collaboration helps solvecross-scale problems by combining peoplersquos knowledgedomains from different scales

23 Multiscale Collaborative VirtualEnvironments

An mCVE is an mVE that supports collaborationIn mCVEs users can choose their own scales in the ob-servation of an object and work together on the sameobject at different scales A team of city planners forexample can have a regular planner of the normal hu-man size at street level and a ldquogiantrdquo planner at the city

Zhang and Furnas 35

level While each planner focuses on planning issues athis or her own level they can help each other on cross-scale issues The giant planner can take the street-levelplanner quickly to a distant destination and the street-level planner can help the partner place buildings orlandmarks at their exact locations Figure 6 shows twoplanners at two scales working together Their cross-scale collaboration could help make a plan that meetsthe requirements at various levels

As seen in Figure 6 each user is represented as anavatar in mCVEs with a scaled size corresponding tothe scale level of the user This avatar metaphor offerstwo benefits First avatars provide rich awareness infor-mation (eg user view locations viewing directionsbody gestures) that are critical to social interactions invirtual environments (Benford Bowers Fahlen Green-halgh amp Snowdon 1995) Second by yoking differentobservation and action parameters under the same ava-tar body avatars can help users better understand theassociation of different observation and action capabili-ties in mCVEs The eye level of a userrsquos avatar tells howfar the user can see the avatarrsquos arm length indicateshow far the user can reach the leg length shows howfast the user can move Seeing the size of a userrsquos avatarother users can tell at what scale the user is working andwhat the user can do

mCVEs could be a promising tool to help people dealwith multiscale structures but this new design paradigmraises many research issues ranging from the conceptualunderstanding of the implications of multiscale for userperception and action to the technical design of

mCVEs to the experimental evaluation of the effective-ness of mCVEs

24 Literature Review

Some research effort has been made to addressdesign issues related to user interaction with large struc-tures Furnas (1986) proposed a theoretical frameworkGeneralized Fisheye Views to manage large hierarchicalstructures by allocating more space for objects of inter-est This technique has been used to visualize varioustypes of data (Sarkar amp Brown 1992 Robertson ampMackinlay 1993 Lamping Rao amp Pirolli 1995 Raabamp Ruger 1996) Koike and Yoshihara (1993) used thefractal concept to expand the tree structure based on thedegrees of interest Cone Trees were built to present filehierarchies (Robertson Mackinlay amp Card 1991)These methods focused only on better using screenspace to deal with the size of information structures butgave little consideration to visualizing structures withdifferent representations based on user interactionscales Multiscale technology (Perlin amp Fox 1993 Bed-erson amp Hollan 1994 Lieberman 1994) providedearly inspiration for this research This technique wasusually regarded as an alternative user interface in 2DThis research extends it into 3D worlds and argues thatmultiscale technology is a powerful design that by al-lowing people to control their observation and actiondomains at different scales can augment our limitedcognitive capabilities

Scaling has been seen in some designs to support useractions in virtual environments Scaling tools were usedto increase reaching distance (Poupyrev BillinghurstWeghorst amp Ichikawa 1996 Mine Brooks amp Sequin1997 Pierce et al 1997) facilitate spatial knowledgeacquisition (Stoakley Conway amp Pausch 1995) andimprove navigation performances (Mackinlay Card ampRobertson 1990) These scaling tools usually focusedon only one interaction parameter and ignored the im-plications of the change of one parameter for all otherrelated perception and action While isolating individualinteraction parameters may work well in some situa-tions uncoordinated perception and action as discussed

Figure 6 Two city planners working at two different scales

36 PRESENCE VOLUME 14 NUMBER 1

previously could cause some problems in the interac-tion with large multiscale structures

Research literature on virtual environments support-ing collaboration is massive but consideration of cross-scale collaboration is rarely seen The most relevantproject is the CALVIN system (Leigh Johnson amp De-Fanti 1996) in which two users can collaborate on adesign project from two different perspectives In thesetwo perspectives objects were rendered with same rep-resentations and the system did not provide users withmultiple levels of abstraction at different scales Alsothe system gave users limited choices of working scalesand could not satisfy their needs for active and interac-tive control over scale across a much broader range

3 Design of mCVEs

There are some design guidelines for 3D collabo-rative virtual environments (CVEs) (Benford SnowdonColebourne OrsquoBrien amp Rodden 1997 EPFL et al1997 Tromp 1999) However they were constructedfor the design of conventional CVEs and common col-laboration actions in CVEs Applying them in designingmCVEs could be difficult because scale issues have notbeen considered in these guidelines In this research weadopt a design approach that starts from understandingthe implications of multiscale technology for user inter-actions and then derives design considerations based onthis understanding In this section efforts are made firstto analyze how multiscale may affect user interactionThen some conceptual designs are proposed to supportuser perception and action in mCVEs

31 Implications of Multiscale for UserInteraction

Multiscale could affect user interactions at differ-ent levels and in different ways Here we first discusshow multiscale can support the understanding of multi-scale structures Then we describe how it may impedeinformation sharing among users at different scales andhow it may facilitate cross-scale action

311 Understanding Multiscale StructuresAs discussed previously interacting with large structuresrequires users to obtain both detailed content and suffi-cient context information This well-known ldquocontent

contextrdquo problem is indeed a multiscale problem be-cause detailed content information and global contextinformation are usually distributed at different scale lev-els To understand complicated structures that presentdifferent features at different scales we usually breakphenomena into components and focus our attentionson issues within certain levels rather than all levels atonce (Pattee 1973 Ahl amp Allen 1996) and thenchoose appropriate tools (eg microscopes telescopes)for observation and analysis at different scales

In virtual environments multiscale technology couldhelp address this ldquofocus contextrdquo problem by allowingusers to control what they want to see and what they cansee from multiscale structures and then to manipulate theamount of context and content information displayed onthe screen Under different interaction scales objects canbe rendered at different sizes tiny objects can be madelarger for more detailed observation and huge objects canbe made smaller for a better big picture The same struc-ture can have different appearances at different scales in-forming users with the rich features of the structure asseen in Figure 5 The understanding of a complicatedstructure could be improved

312 Cross-Scale Information Sharing Col-laboration can facilitate peoplersquos work by improvingcommunications andor cooperation on shared artifacts(Dix 1994) Multiscale collaboration focuses more onartifacts than communications In such artifact-centeredcollaboration users often need to refer to shared arti-facts (Dix) Conventional CVEs are usually ldquoWhat-You-See-Is-What-I-Seerdquo (WYSIWIS) or ldquorelaxed WYSIWISrdquo(Stefik Bobrow Foster Lanning amp Tatar 1987) soknowing what others are referring to is fairly easymCVEs are not like this however Users may have verydifferent views at different scales and see totally differ-ent objects and structures Figure 7 shows the view of acity planner who oversees the whole city (a) and theview of another planner who checks buildings at thestreet level (b) These two views which are from very

Zhang and Furnas 37

different perspectives and show different levels of ab-straction of the city may make it difficult for two plan-ners to understand how their work can be related Aplanner can refer to a particular object in her view butthe other may not see it at all Multiscale technologymay hurt mutual understanding of objects in collabora-tion because of scale-related views

313 Cross-Scale Action Multiscale can alsobring some benefits to collaboration One of them isthat users can leverage their different action capabilitiesand different scopes of influence of their actions Takethe example of the two city planners again When thestreet-level planner needs to go to a distant place thecity-level planner with a great action domain can movethe partner easily to the destination and reduce the tasktime Or when the city-level planner wants to move toan exact location the street-level planner can move thepartnerrsquos avatar accurately because of high manipulationaccuracy It is mentioned previously that a similar tech-nique can be used in object manipulation Thus multi-scale collaboration can improve navigation and manipu-lation activities by allowing users to take advantage ofeach otherrsquos different action capabilities

32 Design Considerations

Since multiscale technology could affect user in-teraction both positively and negatively the design ofmCVEs focuses on maximizing the advantages of multi-scale technology and avoiding the disadvantages Thisprinciple leads to the following design considerations

321 Support for Better Understanding ofMultiscale Structures Multiscale structures shouldbe modeled in such a way that scale-dependent charac-teristics can be observed at different scale levels In thereal world multiscale structures are ubiquitous as seenin Figures 1 and 2 To observe these multiscale struc-tures we need only appropriate tools In 3D virtualworlds however what users can see is predesignedThus to visualize multiscale structures objects have tobe modeled and rendered in a multiscale way

322 Support for Cross-Scale InformationSharing Users should be able to share artifacts ofcommon interest with others across scale Tools such asmultiple views of the world (Gaver Sellen Heath ampLuff 1993) cannot help too much Such techniquesrequire some common objects in different views as ref-erences but usersrsquo very subjective views in mCVEs maymake it difficult to find such references It is a challengeto build mutual understanding of working contexts withsubjective views (Snowdon amp Jaa-Aro 1997) Usersneed help in understanding how their subjective viewsand objects in them are related to each other

323 Support for Cross-Scale Action Usersshould be allowed to get involved in othersrsquo work acrossscale so that they can leverage their different interactioncapabilities in collaboration They should be able to di-rectly manipulate objects that others are working onand move the avatars of other users Of course this in-tervention should be based on mutual agreements toavoid any unwanted consequences In the city-planningexample the city-level planner should not change thingsthe street-level planner is working on without consentOtherwise the street-level planner may get confusedand not understand what has happened

33 Conceptual Design

Based on these considerations some multiscaletools were designed Here discussions focus on designsthat are fundamentally important to mCVEs includingscale-based semantic representations that present scale-related characteristics of multiscale structures dynamic

Figure 7 Two views of a city at two scale levels

38 PRESENCE VOLUME 14 NUMBER 1

view transition that facilitates cross-scale informationsharing and direct avatar manipulation that supportscross-scale action

331 Scale-Based Semantic Representationsfor Multiscale Structures This design renders amultiscale structure with different representations atdifferent scales Figure 5 presents the visual results of asubstance under this technique Initially the substanceappears as a double helix in (a) The user changes thescale to increase the structure size and to see more de-tails in (b) Continuing the scaling the user finds a dif-ferent structure in an atom in (c) The user is informedof semantic characteristics of the substance at differentscales

This scale-based representation technique has oneproblem The change between two very different repre-sentations may confuse users For example the suddenappearance of the structure in Figure 5(c) may distractusers from their primary tasks To address this issue weadopted a fading tool (Bederson amp Hollan 1994Lieberman 1994) to smooth the transition betweendifferent representations Figure 8 shows a transitionprocess from a spherelike molecule from Figure 8(a) toa set of atoms inside it in Figure 8(d) Instead of jump-ing abruptly between different representations the dis-appearance of the molecule and the appearance of atomsare gradual The transition between two representationsis more easily comprehensible Cognition required inunderstanding the change of visual image can be kept atthe level of perceptual processing so that users can freetheir cognitive processing capacities for more compli-cated tasks (Robertson Card amp Mackinlay 1991)

Scale-based semantic representations can be seen asone member of the family of level of detail (LOD) tech-

niques which visualize the same structure with differentgeometric objects according to a particular interactionparameter but with a very different purpose Whilemost LOD techniques such as distance LOD primarilyconcern computation efficiency (Puppo amp Scopigno1997) scale-based semantic representation gives moreconsideration to the userrsquos interaction needs CommonLOD techniques render individual objects with differenttextures or geometries but scale-based semantic repre-sentation visualizes structures with different semanticabstractions LOD techniques can be integrated intoscale-based semantic representations to improve bothuser interaction and machine performance For examplein a city-planning project a city can have different struc-tures at the level of city district and street to informplanners semantically and distance LOD can be appliedin rendering individual objects in each structure to im-prove 3D computation

332 Dynamic View to Support Cross-ScaleInformation Sharing Design here addresses issuesconcerning how to help users at different scales to buildcommon understandings of their divergent views Onedesign choice is to use a dynamic view to narrow downthe view difference We used animation to show thetransition between two views Animation is generatedby interpolating the view positions view orientationsand view scales of two views For example if two usersrsquoviews at two different scales look like Figures 7(a) and7(b) an animation can be generated by adding interme-diate image frames between Figure 7(a) and 7(b) Theanimation can tell how one view can be transformed toanother and how objects in two views are related toeach other

333 Direct Avatar Manipulation to SupportCross-Scale Action To help users get involved inothersrsquo work so that they can take advantage of theirdifferent action capabilities we adopted a design choicethat allows direct manipulation on other usersrsquo avatarsWith such a tool planners in the above city-planningexample can easily cooperate on tasks that require largeaction domain and high action accuracy Of course thisdirect interference should be regulated The user who

Figure 8 Successive fading views in semantic representations

Zhang and Furnas 39

owns the avatar is always aware of such action and hascontrol over whether this action is allowed

The implication of such a direct manipulation of ava-tar body for user interface design deserves more atten-tion Under this design an avatar can ldquoaffordrdquo manipu-lation activities This is quite different from thetraditional role of an avatar merely as user embodimentin conventional CVEs The reason that no direct manip-ulation of an avatar is allowed in conventional CVEsmight be due to the influence of real life where the au-tonomy of human bodies is well respected To betterfacilitate peoplersquos work in virtual environments somereal-life constraints could be relaxed or even abandonedVirtual environments should not be treated as the repli-cation of the real world just as new technologies shouldnot be interpreted by old metaphors Direct manipula-tion of a userrsquos avatar by others is an effort to breakdown the metaphor of reality

4 System Implementation

A desktop mCVE prototype system consisting ofover 20000 lines of Java code was implemented Dis-cussions in this section focus on the choice of toolkits insoftware implementation general system architectureand the implementation of some multiscale tools

41 Choice of Toolkits

The design of virtual environments that supportcollaboration needs to consider issues in three aspects3D graphics rendering network communication man-agement and user interactions (Singhal amp Zyda 1999)It is not our interest to address issues related to low-level graphics rendering or network communicationsOur goal here is to support user interactions with multi-scale structures and with other users at different scalesTo achieve this interaction-oriented design goal weimplemented the mCVE prototype with two toolkitsJava 3D and Java Shared Data Toolkit (JSDT)

This hybrid approach allows a design focus on high-level interaction tools because Java 3D can handlegraphics rendering and JSDT can handle network com-

munications issues With the scene-graph managementof Java 3D we can construct virtual environments andstructures at the level of geometric objects rather thanat the level of vertices and polygons Java 3D allows theextension of new rendering behaviors which is not wellsupported by other CVE toolkits such as DIVE (Carls-son amp Hagsand 1993) and SPLINE (Waters et al1997) This flexibility is critical to the implementationof such tools as scale-based semantic representations anddynamic view transition Furthermore the easy integra-tion of other Java APIs such as Java Swing into Java3D can simplify user interface design JSDT is ideal forthe research because it provides flexible data deliveryservices in support of synchronous collaboration on theInternet JSDT supports basic abstraction of networksessions multicast message communications and con-currency control Thus the design can focus on opti-mizing the management of distributed 3D scenes andcross-scale communications One drawback of this hy-brid approach is the lack of those CVE services seen inSPLINE (Waters et al) Collaboration services have tobe built from scratch

42 System Layer and Architecture

The system layer of the prototype is shown in Fig-ure 9 The implementation focused on the two layersinside the dashed rectangle On top of Java 3D andJSDT are multiscale graphics services and collaboration

Figure 9 System layer of the mCVE prototype

40 PRESENCE VOLUME 14 NUMBER 1

services and above them are multiscale collaborationtools

Multiscale graphics services handle object-renderingissues and update user views based on their view posi-tions and interaction scales Collaboration services de-liver messages among distributed users and manage theconsistency of shared virtual environments Multiscalecollaboration services deal with such collaboration issuesas cross-scale information sharing and cross-scale avatarmanipulation These services lay the foundation for multi-scale tools

43 Implementation of Multiscale Tools

A set of multiscale collaboration tools were imple-mented and integrated into the prototype system Thissection introduces such tools as scaling control whichdistinguishes mCVEs from other CVEs scale-based se-mantic representations which visualize information in amultiscale way cross-scale dynamic view transitionwhich addresses the cross-scale information problemuniquely seen in multiscale collaboration and the man-agement of avatar position in direct avatar manipulationwhich illustrates cross-scale data coordination

431 Scene Graph of the Prototype Systemand Scaling Control The scene graph of the systemis shown in Figure 10 Each Java 3D scene graph has aVirtual Universe object as the root A Locale object at-taches scene graphs to the Virtual Universe objectBranchGroup (BG) nodes collect such objects as geom-etry behavior light sound and so forth A Transform-Group (TG) node defines the coordinates of objectsunder it

The system has two subscene graphs the object scenegraph which is the left child of the Locale object andthe view scene graph which is the right child The ob-ject scene graph collects all objects and behavior nodesThe view scene graph defines the viewing behaviors

To change a userrsquos interaction scale is to change thevalues of the userrsquos interaction parameters relative to thesize of the virtual world Thus scale change can be im-plemented in two ways (1) scaling the virtual world upand down and keeping all interaction parameters un-

touched or (2) changing all interaction parameters andkeeping the virtual world untouched Mathematicallythey are equivalent We adopted the first approach be-cause scaling the virtual world up and down can bedone by modifying just one parameter the scaling fac-tor Si of the top TG in the object scene graph This ismuch simpler than the second approach which involvesthe modification of complicated view parameters

432 Scale-Based Semantic RepresentationsIn the implementation of scale-based semantic represen-tations we adopted a design that uses a delegate objectto wrap all representations and that delivers appropriaterepresentations based on the value of the controllingvariable (Fox 1998) A scale-based semantic representa-tion node is extended from the Java 3D Switch objectEach node has a set of child representations and an or-dered array that defines the scale range for each repre-sentation (Figure 11)

433 Animating View Transition in Collabo-rative View Sharing The key issue in making a view-transition animation is to interpolate two views InmCVEs a view can be uniquely determined by its viewposition P its view orientation O and its view scale Sand be written as V(P O S)

Given two views V0 V(P0 O 0 S0) and V1 V(P1O 1 S1) to implement a view-transition animation is to

Figure 10 Scene graph of the prototype system

Zhang and Furnas 41

choose a path that links V0 and V1 and to set the trajec-tory of viewpoint movement view direction along thepath and observation scale along the path View-pathand view-orientation interpolation techniques are ma-ture (Parent 2001 Watt amp Policarpo 2001) so thisresearch focused only on the interpolation of view scale

A scale-interpolation function could be linear loga-rithmic or in other forms We chose a logarithmic func-tion because it gives users a constant relative rate ofchange in their views Interpolating the view scale loga-rithmically can give users a familiar experience

Assume the animation includes n frames from V0 toV1 The view scale of the ith frame (0 i n) in a log-arithmic interpolation function can be written as

Si Si S0 (1)

where

S S1

S01n

(2)

With corresponding P i and O i from interpolationfunctions for view position and view orientation theview of the ith frame Vi V(P i O i Si) can be ob-tained Figure 12 shows two views seen in Figure 7 asFrame 0 and Frame n and other interpolated viewframes (Due to space limitations we have shown onlyfour frames used for the animation) Showing theseframes successively with a reasonable frame rate can pro-duce an animation linking these two divergent views

434 Cross-Scale Data Consistency and Ava-tar Position in Direct Avatar Manipulation Al-though changing the scaling factor of the top TG in theobject scene graph simplifies the scaling operation such

an approach poses a challenge in cross-scale data consis-tency In CVEs all users usually need to see the sameobject appearing at the same spatial location In conven-tional CVEs this can be easily achieved because an ob-jectrsquos coordinates in different usersrsquo worlds are often thesame In mCVEs however when users are at differentscales their virtual worlds are rescaled according to thescaling factors of the top TG in their own object scenegraphs The same object will have different local coordi-nates in different usersrsquo worlds When the object ismoved by a user the objectrsquos local coordinates in thisuserrsquos world cannot be used to update the objectrsquos newposition in all other usersrsquo worlds

In the implementation we used normalized coordi-nates which are calculated as the ratio of the object lo-cal coordinates in the userrsquos world to the userrsquos scaleLet Pl(xl yl zl) and Sl denote the local coordinates andthe userrsquos scale respectively The normalized coordi-nates Pn can be written as

Pn PlSl (3)

By doing so the same object will always have the samePn This Pn is used to define the coordinates of the ob-ject in every userrsquos object scene graph Modifying thespatial location of an object in a userrsquos world leads to anew Pn All users can use this Pn to update the coordi-nates of the object in their object scene graphs There-fore Pn enforces data consistency across scale Becausethe scaling factor is on the top of the scene graph theobjectrsquos final location in a userrsquos world is Pl the productof Pn and this userrsquos scale Sl With different Sl users willbe able to see the world rendered at different scales

An avatarrsquos position can be managed similarly butwith a small difference An avatar object differs fromother ordinary objects in a way that in its host userrsquos

Figure 11 Scale-based semantic representation node

Figure 12 Frames of a dynamic view-transition animation

42 PRESENCE VOLUME 14 NUMBER 1

world the coordinates of the avatar are affiliated withthe view scene graph rather than the object scenegraph so that the avatar always moves with the view Inother usersrsquo worlds however the avatar object is underthe object scene graphs Figure 13(a) is the scene graphof the world of a user say A who interacts with the vir-tual environment at a location Pa and with a scale SaFigure 13(b) is the scene graph of the world of anotheruser say B User Arsquos avatar is in User Brsquos object graphBecause Pa is not governed by Sa the normalized coor-dinates of the avatar in User Brsquos object graph should bewritten as

PnPaSa (4)

The body size of User Arsquos avatar changes with thescale of User A so a scale parameter is also needed todefine the size of User Arsquos avatar in User Brsquos objectscene graph so that User B can be aware of User Arsquosscale This scale should also be normalized Let Sn de-note the normalized scale where Sn is simply the recip-rocal of Sa

Sn 1Sa (5)

This is because the avatar body changes inversely pro-portional to the scale factor Sa (shrinking the world isequivalent to growing the avatar body and growing theworld is equivalent to shrinking the avatar body)

The implementation of other cross-scale tools includ-ing direct avatar manipulation also uses Pn and Sn todefine the coordinates and size of an avatar entity Thisapproach guarantees the consistency of the avatar posi-tion and size in all usersrsquo views at different scales

5 Experimental Evaluation

We evaluated multiscale collaboration in subjecttests An experiment was designed to assess the benefitsusers can get from an mCVE in accomplishing tasks re-quiring cross-scale coordination

51 Experiment Design and Procedure

Subject tasks in the experiment involved searchingfor a ldquobombrdquo on a square ground plane (2000 2000m2) with a distinctly shaped building in each corner(square hexagon octagon and circle as seen fromabove) Each had a height of 12 m and a base of about6400 m2 On the ground behind each building was aunit cube (1 m3) containing a unique text name andsmiley face (about 05 05 m2) The bomb was insideone of these four cubes In the test subjects were placedin the middle of the square plane They had a defaulteye level of 168 m and a default moving step of 1 m inthe virtual environment The shape of the building thatthe bomb was nearby was known in advance and sub-jects needed to find that building locate the bomb boxand key in the name of the box to defuse the bomb

A 2 2 2 factorial design that has six treatmentstwo noncollaboration and four collaboration wasadopted (Table 1) For the two noncollaboration treat-ments VE is just a conventional 3D virtual environ-ment and M-VE is a VE enhanced by multiscale toolsspecifically it allows users to change their interactionscales eye level and speed Among four collaborationtreatments CVE a conventional collaborative virtualenvironment is the only one without multiscale toolsThe other three treatments all equipped with multiscale

Figure 13 Two scene graphs for the same avatar

Table 1 Six Treatments in a 2 2 2 Design

Noncollaboration Collaboration

Nonmultiscale VE CVEMultiscale M-VE NR

GUIDEMOVE

Zhang and Furnas 43

tools differ in the assignment of subject task roles (be-ing a giant or a normal person) and in the way subjectsaffect each otherrsquos work across scales In one treatmentthe roles of subjects are not predefined and subjects canchoose their own interaction scales as desired This envi-ronment is labeled as NR (No Role) The other twomCVEs both assign one subject to be a giant and theother a normal person so that subjects can change theirinteraction scales only within a limited range In onesuch environment the giant is permitted to move thepartner directly and this treatment is labeled as MOVEAnother condition allows the giant to guide the move-ment of the partner only verbally and is denoted asGUIDE

Recruited through email 24 students paired in 12groups participated in the experiment In noncollabora-tion treatments they worked on their own and in col-laboration treatments they communicated through anaudio channel The performances were measured bytask-completion time

52 Results

An ANOVA analysis of data from four treatments(VE M-VE CVE and MOVE) shows main effects ofboth collaboration (F170 1298 p 0001) andmultiscale (F170 7090 p 00001) Interaction isnot significant (F170 187 p 176) as seen in Fig-ure 14 Subjects performed best in MOVE where theycould take full advantage of multiscale and collabora-tion and they did worst in VE where there was no helpat all

Subjects used different strategies in the different con-ditions Without multiscale tools subjects had to goaround and count the number of sides of all four build-ings to find the target a very time-consuming processWith multiscale subjects could increase their sizes to seebuilding shapes from above and approach the bombquickly so the time required was be reduced signifi-cantly

The performance difference among the three mCVEsalso indicates the importance of different collaborationsupports In the NR treatment subjects had to assumetheir roles via an expensive negotiation process In the

GUIDE treatment the giant subject who could see thebuilding shape must coordinate with the partner aboutnavigation verbally through a costly grounding process(Clark amp Brennan 1991) In MOVE however the all-seeing giant could actually move the partner to the des-tination quickly and bypassed grounding processes

6 Discussion

In this paper we have introduced a novel interac-tive environment multiscale collaborative virtual envi-ronment which gives users the freedom to control theirinteraction scales in collaboration Multiscale collabora-tion can help better allocate scarce scale resources andcombine different expertise and action capabilities ofindividual users in their work with large structures Wehave also discussed the implications of multiscale tech-nology for user interaction and the design and develop-ment of a Java-based mCVE system Our subject testshave shown that multiscale collaboration can facilitatepeoplersquos work that requires cross-scale information andcollaboration

Research reported in this paper has largely focused ongeneral interaction issues in mCVEs Designs have beenlimited to generic tools to support multiscale collabora-tion and experimental study has focused on generaltasks We expect that multiscale collaboration tools canbe used to support peoplersquos specific work in the realworld To achieve this goal the design of multiscale

Figure 14 Time (in seconds) for the six treatments

44 PRESENCE VOLUME 14 NUMBER 1

tools should support domain-specific tasks One of ourongoing research projects focuses on studying how multi-scale collaboration can support research in materials sci-ence and engineering We are investigating the needs ofmaterials scientists for multiscale collaboration and willdesign and implement tools that support these needsThe effectiveness and usability of these tools will also beevaluated when they are ready

At the same time we are interested in behavior issuesin mCVEs Some important research questions concern-ing user behavior in mCVEs have not been studiedthoroughly It is still unclear when and how users canbenefit from collaborative object manipulation acrossscale whether the designed dynamic view-transitiontechnique can indeed improve cross-scale informationsharing what mechanism is needed to help users bettermanage cross-scale collaboration overheads and soforth In addition it has been found that multiscale alsobrings some new challenges in social interactions(Zhang amp Furnas 2002) and efforts are needed tostudy what can be done to address these interaction is-sues Research in this direction will deepen our under-standing of mCVEs and help to design better tools tosupport multiscale collaboration

Acknowledgments

This research was funded in part by Microsoft Research

References

Ahl V amp Allen T (1996) Hierarchy theory A vision vocab-ulary and epistemology New York Columbia UniversityPress

Alexander C Ishikawa S amp Silverstein M (1977) A pat-tern language Oxford UK Oxford University Press

Bederson B amp Hollan J (1994) Pad A zoominggraphical interface for exploring alternate interface physicsProceedings of ACM UIST rsquo94 17ndash26

Benford S Bowers J Fahlen L Greenhalgh C amp Snow-don D (1995) User embodiment in collaborative virtualenvironments Proceedings of ACM CHI rsquo95 242ndash249

Benford S Snowdon D Colebourne A OrsquoBrien J ampRodden T (1997) Informing the design of collaborativevirtual environments Proceedings of ACM GROUP rsquo97 71ndash80

Carlsson C amp Hagsand O (1993) DIVEmdashA platform formulti-user virtual environments Computer and Graphics17(6) 663ndash669

Clark H amp Brennan S (1991) Grounding in communica-tion In L B Resnick J Levine amp S D Behrend (Eds)Perspectives on socially shared cognition (pp 127ndash149)Washington DC American Psychological Association

Dix A J (1994) Computer-supported cooperative workmdashAframework In D Rosenburg amp C Hutchison (Eds) De-sign Issues in CSCW (pp 23ndash37) Berlin Springer-Verlag

EPFL Geneva IIS TNO Lancaster UCL et al (1997)Guidelines for building CVE applications COVEN ProjectRetrieved November 21 2002 from httpwwwcsuclacukstaffmslatervrCovenPublicDeliverablesdel26PDF

Fox D (1998) Tabula rasa A multi-scale user interface sys-tem Unpublished doctoral dissertation New York Univer-sity

Furnas G (1986) Generalized fisheye views Proceedings ofACM CHI rsquo86 16ndash23

Gaver W Sellen A Heath C amp Luff P (1993) One isnot enough Multiple views in a media space Proceedings ofACM INTERCHI rsquo93 335ndash341

Hollan J amp Stornetta S (1992) Beyond being there Pro-ceedings of ACM CHI rsquo92 119ndash125

Koike H amp Yoshihara H (1993) Fractal approaches forvisualizing huge hierarchies Proceedings of IEEE VL rsquo9355ndash60

Lamping J Rao R amp Pirolli P (1995) A focus contexttechnique based on hyperbolic geometry for visualizinglarge hierarchies Proceedings of ACM CHI rsquo95 401ndash408

Leigh J Johnson A amp DeFanti T (1996) CALVIN Animmersimedia design environment utilizing heterogeneousperspectives Proceedings of IEEE International Conferenceon Multimedia Computing and Systems 20ndash23

Lieberman H (1994) Powers of ten thousand Navigating inlarge information spaces Proceedings of ACM UIST rsquo9415ndash16

Mackinlay J Card S amp Robertson G (1990) Rapid con-trolled movement through a virtual 3D workspace Proceed-ings of ACM SIGGRAPH rsquo90 171ndash176

Mine M Brooks F amp Sequin C (1997) Moving objects

Zhang and Furnas 45

in space Exploiting proprioception in virtual-environmentinteraction Proceedings of ACM SIGGRAPH rsquo97 19ndash26

Morrison P amp Morrison P (1994) Powers of ten About therelative size of things in the universe and the effect of addinganother zero San Francisco W H Freeman

Parent R (2001) Computer animation Algorithms and tech-niques San Francisco Morgan-Kaufmann

Pattee H (1973) Hierarchy theory The challenge of complexsystems New York George Braziller

Perlin K amp Fox D (1993) Pad An alternative approach tothe computer interface Proceedings of ACM SIGGRAPHrsquo93 57ndash64

Pierce J Forsberg A Conway M Hong S Zeleznik Ramp Mine M (1997) Image plane interaction techniques in3D immersive environments Proceedings of the 1997 Sympo-sium on Interactive 3D Graphics 39ndash43

Poupyrev I Billinghurst M Weghorst S amp Ichikawa T(1996) The go-go interaction technique Non-linear map-ping for direct manipulation in VR Proceedings of ACMUIST rsquo96 79ndash80

Puppo E amp Scopigno R (1997) Simplification LOD multi-resolution Principles and applications Proceedings ofEUROGRAPHICS 16 (3)

Raab A amp Ruger M (1996) 3D-Zoom Interactive visual-ization of structures and relations in complex graphics InB Girod H Nieman amp H Seidel (Eds) 3D image analy-sis and synthesis (pp 125ndash132) Sankt Augustin GermanyInfix-Verlag

Robbins M (2001) Modeling and simulation of the failureand toughness of polymersubstrate bonds The second An-nual Multiscale ModelingmdashBridging the Gap Between Atom-istic and Meso Scales

Robertson G Card S amp Mackinlay J (1991) Informationvisualization using 3D interactive animation Proceedings ofACM CHI rsquo91 461ndash462

Robertson G amp Mackinlay J (1993) The document lensProceedings of ACM UIST rsquo93 101ndash110

Robertson G Mackinlay J amp Card S (1991) Cone treesAnimated 3D visualizations of hierarchical informationCommunications of the ACM 34 (2) p 89ndash194

Sarkar M amp Brown M H (1992) Graphical fisheye viewsof graphs Proceedings of ACM CHI rsquo92 83ndash91

Singhal S amp Zyda M (1999) Networked virtual environ-ments Design and implementation New York ACM Press

Snowdon D amp Jaa-Aro K (1997) A subjective virtual envi-ronment for collaborative information visualization VirtualReality Universe rsquo97 Retrieved May 23 2003 fromhttpwwwcyberedgecomvru_paperssnowdenhtm

Stappers P Gaver W amp Overbeeke C (2000) Beyond thelimits of real-time realism 2000 Chapter to appear in L JHettinger amp M W Haas (Eds) Psychological issues in thedesign and use of virtual and adaptive environment Mah-wah NJ Erlbaum

Stefik M Bobrow D Foster G Lanning S amp Tatar D(1987) WYSIWIS revised Early experiences with multiuserinterfaces ACM Transactions on Information Systems(TOIS) 5(2) 147ndash167

Stoakley R Conway M amp Pausch R (1995) Virtual real-ity on a WIM Interactive worlds in miniature Proceedingsof ACM CHI rsquo95 265ndash272

Sutherland I (1965) The ultimate display Proceedings ofIFIP Congress 2

Tromp J G (1999) Usability design for CVEs COVENProject Retrieved November 21 2002 from httpcovenlancsacuk4deliverablesdel29pdf

Waters R Anderson D Barrus J Brogan D Casey MMcKeown S et al (1997) Diamond park and SPLINESocial virtual reality with 3D animation spoken interactionand runtime extendibility Presence Teleoperators and Vir-tual Environments 6(4) 461ndash481

Watt A amp Policarpo F (2001) 3D games Animation andadvanced real-time rendering New York Addison-Wesley

Zhang X amp Furnas G (2002) Social interactions in multi-scale CVEs Proceedings of ACM CVE 2002 31ndash38

46 PRESENCE VOLUME 14 NUMBER 1

tion as seen in Figure 3 but making sense of the rela-tionship between these different maps across scales maydemand considerable cognitive resources for memoryand information processing Users need help controllingthe scale levels of observation and analysis

To address these multiscale issues we propose a newtype of interaction environment the multiscale collabo-rative virtual environment (mCVE) Multiscale technol-ogy allows people to control their observation and ac-tion domains dynamically Collaboration technologybenefits people because of the division of labor and theparallel working processes Combining multiscale andcollaboration helps to better allocate individual usersrsquocognitive resources to different scales and then exploitthe resulting different perception and action capabilitiesThis multiscale collaboration technique is discussed inthe context of 3D virtual environments which can visu-alize more complicated structures mCVEs allow usersto work at different scale levels being the size of ants toobserve fine details and maintain high action accuracy

and being the size of giants to see the big picture ofstructures and have a broad interaction range By collab-orating from different scale levels in mCVEs users canbridge the scale barrier

This paper begins with the introduction of mCVEsand literature review Then it advances to design issuesrelated to the visualization of multiscale structurescross-scale information sharing and cross-scale actionAfter a discussion of the implementation of an mCVEprototype system the paper describes an experimentalstudy of the effectiveness of multiscale collaboration

2 Multiscale Collaborative VirtualEnvironments

The design of mCVEs deviates from the path setby Ivan Sutherland which is to make computer-simulated worlds real (Sutherland 1965) By going be-yond being real (Hollan amp Stornetta 1992 Stappers

Figure 3 Maps of a US city at different scales (Images provided by wwwmapquestcom)

Zhang and Furnas 33

Gaver amp Overbeeke 2000) we can expand designspace and find new ways to address old problems Thissection discusses the features of multiscale virtual envi-ronments (mVEs) the necessity of multiscale collabora-tion and the benefits of mCVEs Relevant literature isalso reviewed

21 Multiscale Virtual Environments

The absence of real-world physics in virtual envi-ronments has led to some innovative designs in virtualenvironments Teleportation for example is a usefulnavigation tool that transcends speed and time AnmVE is a design that overcomes the scale barrier andallows users to manipulate the scale of the virtual spacein their work

211 Scale in Virtual Environments In com-puter graphics scale usually refers to the rendered sizeof objects To manipulate the scale of a virtual environ-ment is to resize the virtual environment Figure 4shows images of a DNA structure at two scales Thesetwo images differ only in rendered sizes and resultingdifferent levels of detail However if the scale differenceis more dramatic image difference can go beyond geo-metric size as seen in Figure 5 where three images atsignificantly different scales show very different and im-portant characteristics the double helix in (a) the mo-lecular structure in (b) and the nucleus and the electroncloud in an atom in (c)

212 Scale in Multiscale Virtual Environ-ments This scale-dependent rendering technique dis-tinguishes mVEs from conventional virtual environ-ments Conventional virtual environments renderobjects with the same representation regardless of userinterests mVEs render objects with different representa-tions corresponding to different scale ranges In mVEsusers can easily change their observation scales and re-ceive different representations from big pictures to localdetails

Another important but less obvious characteristic ofmVEs is that users also get different interaction do-mains To change the scale of a virtual environment isto shift a userrsquos action capabilities One action can leadto different results at different scales For example auserrsquos movement could be at the nanometer level whenthe user is at the atomic scale and the same movementwould be at the micrometer level when the user movesto the molecular scale When the user observes a virtualplanet system the movement scale can be at the level ofhundreds of thousands of kilometers In mVEs userobservation and action capabilities are coordinated atthe same or comparable scale level

Users need this association in mVEs so that they canwork appropriately with what they see For examplewhen users observe an object structure at the level of 1mm their movement scale should be at a comparablescale say 1 mmsec Moving at a very different scalesay 1 kmsec would make it hard to keep a consistentview Objects of interest would be constantly speedingout of sight Conversely if the structure of interest is atthe level of 1 km moving at a scale of 1 mmsec would

Figure 4 DNA rendered at two different scales

Figure 5 Successive views of a DNA structure rendered at different

scales Perspective lines added for the figure to show where each

successive view comes from

34 PRESENCE VOLUME 14 NUMBER 1

be problematic It would demand tremendous effort tochange the view of the large structure even slightlyOther interaction parameters such as reachable distanceand manipulation accuracy should be similarly coordi-nated

Thus mVEs provide users with new interaction capa-bilities that are usually unavailable in conventional vir-tual environments In mVEs users can interact withvirtual worlds at multiple scale levels At any given scalesome things will be easy to see and manipulate whileothers will be either too large or too small to displayChanging the scale moves new things into the easilyvisible and manipulative range Coordinated observationand action capabilities help users to better interact withmultiscale structures Multiscale extends the designspace for information visualization and user interaction

22 Multiscale Collaboration

Multiscale collaboration can further facilitate userinteraction with multiscale structures In particular itcan help people with work on large structures that needmore labor on rapidly changing structures that demandparallel working and on complex structures that requiredomain knowledge beyond what an individual mayhave

221 Large Multiscale Structures Managinga large multiscale structure could be difficult for a singleuser Take a city-planning example Adding new designelements (eg paths or landmarks) may force plannersto move back and forth frequently between differentscales and check the impact of new additions on designat different levels For a large city continuously chang-ing scale could be very demanding and costly if there isjust one planner Multiscale collaboration can offersome help by placing individual planners with differentscales and having them focus on issues at their own scalelevel Planners can then work independently on prob-lems within their own scale scopes and collaborativelydeal with cross-scale issues Collaboration divides bigcross-scale problems into smaller within-scale ones thatcan be more easily handled by individuals

222 Rapidly Changing Multiscale StructuresIt could also be a challenge for a user to manage a struc-ture that changes rapidly however small that structuremay be Consider a scenario in which a small computernetwork managed by a system administrator is beinginfected by a worm virus The system administrator maynot be able to keep pace with the speed at which thevirus spreads If more system administrators are in-volved they can have individuals to check subnets atdifferent levels simultaneously and then take appropri-ate actions to stop the virus from further damaging theentire network Collaboration by supporting parallelworking improves peoplersquos response to problems atdifferent levels

223 Cross-Domain Multiscale StructuresChallenges may also come from the diversified knowl-edge and expertise required in understanding and man-aging structures spanning across the boundaries of sci-entific domains Understanding cross-domain multiscalestructures could be daunting for individuals For exam-ple in studying the strength of a new material acrossdifferent scale levels no single person would possess allthe required in-depth knowledge of chemistry materialsscience and mechanical engineering Chemists materi-als scientists and mechanical engineers may need towork together on this issue While they can focus onproblems in their own expertise domains individuallyworking together helps them see how properties at dif-ferent scale levels may affect each other and even dis-cover important issues that might be ignored withoutcross-scale collaboration Collaboration helps solvecross-scale problems by combining peoplersquos knowledgedomains from different scales

23 Multiscale Collaborative VirtualEnvironments

An mCVE is an mVE that supports collaborationIn mCVEs users can choose their own scales in the ob-servation of an object and work together on the sameobject at different scales A team of city planners forexample can have a regular planner of the normal hu-man size at street level and a ldquogiantrdquo planner at the city

Zhang and Furnas 35

level While each planner focuses on planning issues athis or her own level they can help each other on cross-scale issues The giant planner can take the street-levelplanner quickly to a distant destination and the street-level planner can help the partner place buildings orlandmarks at their exact locations Figure 6 shows twoplanners at two scales working together Their cross-scale collaboration could help make a plan that meetsthe requirements at various levels

As seen in Figure 6 each user is represented as anavatar in mCVEs with a scaled size corresponding tothe scale level of the user This avatar metaphor offerstwo benefits First avatars provide rich awareness infor-mation (eg user view locations viewing directionsbody gestures) that are critical to social interactions invirtual environments (Benford Bowers Fahlen Green-halgh amp Snowdon 1995) Second by yoking differentobservation and action parameters under the same ava-tar body avatars can help users better understand theassociation of different observation and action capabili-ties in mCVEs The eye level of a userrsquos avatar tells howfar the user can see the avatarrsquos arm length indicateshow far the user can reach the leg length shows howfast the user can move Seeing the size of a userrsquos avatarother users can tell at what scale the user is working andwhat the user can do

mCVEs could be a promising tool to help people dealwith multiscale structures but this new design paradigmraises many research issues ranging from the conceptualunderstanding of the implications of multiscale for userperception and action to the technical design of

mCVEs to the experimental evaluation of the effective-ness of mCVEs

24 Literature Review

Some research effort has been made to addressdesign issues related to user interaction with large struc-tures Furnas (1986) proposed a theoretical frameworkGeneralized Fisheye Views to manage large hierarchicalstructures by allocating more space for objects of inter-est This technique has been used to visualize varioustypes of data (Sarkar amp Brown 1992 Robertson ampMackinlay 1993 Lamping Rao amp Pirolli 1995 Raabamp Ruger 1996) Koike and Yoshihara (1993) used thefractal concept to expand the tree structure based on thedegrees of interest Cone Trees were built to present filehierarchies (Robertson Mackinlay amp Card 1991)These methods focused only on better using screenspace to deal with the size of information structures butgave little consideration to visualizing structures withdifferent representations based on user interactionscales Multiscale technology (Perlin amp Fox 1993 Bed-erson amp Hollan 1994 Lieberman 1994) providedearly inspiration for this research This technique wasusually regarded as an alternative user interface in 2DThis research extends it into 3D worlds and argues thatmultiscale technology is a powerful design that by al-lowing people to control their observation and actiondomains at different scales can augment our limitedcognitive capabilities

Scaling has been seen in some designs to support useractions in virtual environments Scaling tools were usedto increase reaching distance (Poupyrev BillinghurstWeghorst amp Ichikawa 1996 Mine Brooks amp Sequin1997 Pierce et al 1997) facilitate spatial knowledgeacquisition (Stoakley Conway amp Pausch 1995) andimprove navigation performances (Mackinlay Card ampRobertson 1990) These scaling tools usually focusedon only one interaction parameter and ignored the im-plications of the change of one parameter for all otherrelated perception and action While isolating individualinteraction parameters may work well in some situa-tions uncoordinated perception and action as discussed

Figure 6 Two city planners working at two different scales

36 PRESENCE VOLUME 14 NUMBER 1

previously could cause some problems in the interac-tion with large multiscale structures

Research literature on virtual environments support-ing collaboration is massive but consideration of cross-scale collaboration is rarely seen The most relevantproject is the CALVIN system (Leigh Johnson amp De-Fanti 1996) in which two users can collaborate on adesign project from two different perspectives In thesetwo perspectives objects were rendered with same rep-resentations and the system did not provide users withmultiple levels of abstraction at different scales Alsothe system gave users limited choices of working scalesand could not satisfy their needs for active and interac-tive control over scale across a much broader range

3 Design of mCVEs

There are some design guidelines for 3D collabo-rative virtual environments (CVEs) (Benford SnowdonColebourne OrsquoBrien amp Rodden 1997 EPFL et al1997 Tromp 1999) However they were constructedfor the design of conventional CVEs and common col-laboration actions in CVEs Applying them in designingmCVEs could be difficult because scale issues have notbeen considered in these guidelines In this research weadopt a design approach that starts from understandingthe implications of multiscale technology for user inter-actions and then derives design considerations based onthis understanding In this section efforts are made firstto analyze how multiscale may affect user interactionThen some conceptual designs are proposed to supportuser perception and action in mCVEs

31 Implications of Multiscale for UserInteraction

Multiscale could affect user interactions at differ-ent levels and in different ways Here we first discusshow multiscale can support the understanding of multi-scale structures Then we describe how it may impedeinformation sharing among users at different scales andhow it may facilitate cross-scale action

311 Understanding Multiscale StructuresAs discussed previously interacting with large structuresrequires users to obtain both detailed content and suffi-cient context information This well-known ldquocontent

contextrdquo problem is indeed a multiscale problem be-cause detailed content information and global contextinformation are usually distributed at different scale lev-els To understand complicated structures that presentdifferent features at different scales we usually breakphenomena into components and focus our attentionson issues within certain levels rather than all levels atonce (Pattee 1973 Ahl amp Allen 1996) and thenchoose appropriate tools (eg microscopes telescopes)for observation and analysis at different scales

In virtual environments multiscale technology couldhelp address this ldquofocus contextrdquo problem by allowingusers to control what they want to see and what they cansee from multiscale structures and then to manipulate theamount of context and content information displayed onthe screen Under different interaction scales objects canbe rendered at different sizes tiny objects can be madelarger for more detailed observation and huge objects canbe made smaller for a better big picture The same struc-ture can have different appearances at different scales in-forming users with the rich features of the structure asseen in Figure 5 The understanding of a complicatedstructure could be improved

312 Cross-Scale Information Sharing Col-laboration can facilitate peoplersquos work by improvingcommunications andor cooperation on shared artifacts(Dix 1994) Multiscale collaboration focuses more onartifacts than communications In such artifact-centeredcollaboration users often need to refer to shared arti-facts (Dix) Conventional CVEs are usually ldquoWhat-You-See-Is-What-I-Seerdquo (WYSIWIS) or ldquorelaxed WYSIWISrdquo(Stefik Bobrow Foster Lanning amp Tatar 1987) soknowing what others are referring to is fairly easymCVEs are not like this however Users may have verydifferent views at different scales and see totally differ-ent objects and structures Figure 7 shows the view of acity planner who oversees the whole city (a) and theview of another planner who checks buildings at thestreet level (b) These two views which are from very

Zhang and Furnas 37

different perspectives and show different levels of ab-straction of the city may make it difficult for two plan-ners to understand how their work can be related Aplanner can refer to a particular object in her view butthe other may not see it at all Multiscale technologymay hurt mutual understanding of objects in collabora-tion because of scale-related views

313 Cross-Scale Action Multiscale can alsobring some benefits to collaboration One of them isthat users can leverage their different action capabilitiesand different scopes of influence of their actions Takethe example of the two city planners again When thestreet-level planner needs to go to a distant place thecity-level planner with a great action domain can movethe partner easily to the destination and reduce the tasktime Or when the city-level planner wants to move toan exact location the street-level planner can move thepartnerrsquos avatar accurately because of high manipulationaccuracy It is mentioned previously that a similar tech-nique can be used in object manipulation Thus multi-scale collaboration can improve navigation and manipu-lation activities by allowing users to take advantage ofeach otherrsquos different action capabilities

32 Design Considerations

Since multiscale technology could affect user in-teraction both positively and negatively the design ofmCVEs focuses on maximizing the advantages of multi-scale technology and avoiding the disadvantages Thisprinciple leads to the following design considerations

321 Support for Better Understanding ofMultiscale Structures Multiscale structures shouldbe modeled in such a way that scale-dependent charac-teristics can be observed at different scale levels In thereal world multiscale structures are ubiquitous as seenin Figures 1 and 2 To observe these multiscale struc-tures we need only appropriate tools In 3D virtualworlds however what users can see is predesignedThus to visualize multiscale structures objects have tobe modeled and rendered in a multiscale way

322 Support for Cross-Scale InformationSharing Users should be able to share artifacts ofcommon interest with others across scale Tools such asmultiple views of the world (Gaver Sellen Heath ampLuff 1993) cannot help too much Such techniquesrequire some common objects in different views as ref-erences but usersrsquo very subjective views in mCVEs maymake it difficult to find such references It is a challengeto build mutual understanding of working contexts withsubjective views (Snowdon amp Jaa-Aro 1997) Usersneed help in understanding how their subjective viewsand objects in them are related to each other

323 Support for Cross-Scale Action Usersshould be allowed to get involved in othersrsquo work acrossscale so that they can leverage their different interactioncapabilities in collaboration They should be able to di-rectly manipulate objects that others are working onand move the avatars of other users Of course this in-tervention should be based on mutual agreements toavoid any unwanted consequences In the city-planningexample the city-level planner should not change thingsthe street-level planner is working on without consentOtherwise the street-level planner may get confusedand not understand what has happened

33 Conceptual Design

Based on these considerations some multiscaletools were designed Here discussions focus on designsthat are fundamentally important to mCVEs includingscale-based semantic representations that present scale-related characteristics of multiscale structures dynamic

Figure 7 Two views of a city at two scale levels

38 PRESENCE VOLUME 14 NUMBER 1

view transition that facilitates cross-scale informationsharing and direct avatar manipulation that supportscross-scale action

331 Scale-Based Semantic Representationsfor Multiscale Structures This design renders amultiscale structure with different representations atdifferent scales Figure 5 presents the visual results of asubstance under this technique Initially the substanceappears as a double helix in (a) The user changes thescale to increase the structure size and to see more de-tails in (b) Continuing the scaling the user finds a dif-ferent structure in an atom in (c) The user is informedof semantic characteristics of the substance at differentscales

This scale-based representation technique has oneproblem The change between two very different repre-sentations may confuse users For example the suddenappearance of the structure in Figure 5(c) may distractusers from their primary tasks To address this issue weadopted a fading tool (Bederson amp Hollan 1994Lieberman 1994) to smooth the transition betweendifferent representations Figure 8 shows a transitionprocess from a spherelike molecule from Figure 8(a) toa set of atoms inside it in Figure 8(d) Instead of jump-ing abruptly between different representations the dis-appearance of the molecule and the appearance of atomsare gradual The transition between two representationsis more easily comprehensible Cognition required inunderstanding the change of visual image can be kept atthe level of perceptual processing so that users can freetheir cognitive processing capacities for more compli-cated tasks (Robertson Card amp Mackinlay 1991)

Scale-based semantic representations can be seen asone member of the family of level of detail (LOD) tech-

niques which visualize the same structure with differentgeometric objects according to a particular interactionparameter but with a very different purpose Whilemost LOD techniques such as distance LOD primarilyconcern computation efficiency (Puppo amp Scopigno1997) scale-based semantic representation gives moreconsideration to the userrsquos interaction needs CommonLOD techniques render individual objects with differenttextures or geometries but scale-based semantic repre-sentation visualizes structures with different semanticabstractions LOD techniques can be integrated intoscale-based semantic representations to improve bothuser interaction and machine performance For examplein a city-planning project a city can have different struc-tures at the level of city district and street to informplanners semantically and distance LOD can be appliedin rendering individual objects in each structure to im-prove 3D computation

332 Dynamic View to Support Cross-ScaleInformation Sharing Design here addresses issuesconcerning how to help users at different scales to buildcommon understandings of their divergent views Onedesign choice is to use a dynamic view to narrow downthe view difference We used animation to show thetransition between two views Animation is generatedby interpolating the view positions view orientationsand view scales of two views For example if two usersrsquoviews at two different scales look like Figures 7(a) and7(b) an animation can be generated by adding interme-diate image frames between Figure 7(a) and 7(b) Theanimation can tell how one view can be transformed toanother and how objects in two views are related toeach other

333 Direct Avatar Manipulation to SupportCross-Scale Action To help users get involved inothersrsquo work so that they can take advantage of theirdifferent action capabilities we adopted a design choicethat allows direct manipulation on other usersrsquo avatarsWith such a tool planners in the above city-planningexample can easily cooperate on tasks that require largeaction domain and high action accuracy Of course thisdirect interference should be regulated The user who

Figure 8 Successive fading views in semantic representations

Zhang and Furnas 39

owns the avatar is always aware of such action and hascontrol over whether this action is allowed

The implication of such a direct manipulation of ava-tar body for user interface design deserves more atten-tion Under this design an avatar can ldquoaffordrdquo manipu-lation activities This is quite different from thetraditional role of an avatar merely as user embodimentin conventional CVEs The reason that no direct manip-ulation of an avatar is allowed in conventional CVEsmight be due to the influence of real life where the au-tonomy of human bodies is well respected To betterfacilitate peoplersquos work in virtual environments somereal-life constraints could be relaxed or even abandonedVirtual environments should not be treated as the repli-cation of the real world just as new technologies shouldnot be interpreted by old metaphors Direct manipula-tion of a userrsquos avatar by others is an effort to breakdown the metaphor of reality

4 System Implementation

A desktop mCVE prototype system consisting ofover 20000 lines of Java code was implemented Dis-cussions in this section focus on the choice of toolkits insoftware implementation general system architectureand the implementation of some multiscale tools

41 Choice of Toolkits

The design of virtual environments that supportcollaboration needs to consider issues in three aspects3D graphics rendering network communication man-agement and user interactions (Singhal amp Zyda 1999)It is not our interest to address issues related to low-level graphics rendering or network communicationsOur goal here is to support user interactions with multi-scale structures and with other users at different scalesTo achieve this interaction-oriented design goal weimplemented the mCVE prototype with two toolkitsJava 3D and Java Shared Data Toolkit (JSDT)

This hybrid approach allows a design focus on high-level interaction tools because Java 3D can handlegraphics rendering and JSDT can handle network com-

munications issues With the scene-graph managementof Java 3D we can construct virtual environments andstructures at the level of geometric objects rather thanat the level of vertices and polygons Java 3D allows theextension of new rendering behaviors which is not wellsupported by other CVE toolkits such as DIVE (Carls-son amp Hagsand 1993) and SPLINE (Waters et al1997) This flexibility is critical to the implementationof such tools as scale-based semantic representations anddynamic view transition Furthermore the easy integra-tion of other Java APIs such as Java Swing into Java3D can simplify user interface design JSDT is ideal forthe research because it provides flexible data deliveryservices in support of synchronous collaboration on theInternet JSDT supports basic abstraction of networksessions multicast message communications and con-currency control Thus the design can focus on opti-mizing the management of distributed 3D scenes andcross-scale communications One drawback of this hy-brid approach is the lack of those CVE services seen inSPLINE (Waters et al) Collaboration services have tobe built from scratch

42 System Layer and Architecture

The system layer of the prototype is shown in Fig-ure 9 The implementation focused on the two layersinside the dashed rectangle On top of Java 3D andJSDT are multiscale graphics services and collaboration

Figure 9 System layer of the mCVE prototype

40 PRESENCE VOLUME 14 NUMBER 1

services and above them are multiscale collaborationtools

Multiscale graphics services handle object-renderingissues and update user views based on their view posi-tions and interaction scales Collaboration services de-liver messages among distributed users and manage theconsistency of shared virtual environments Multiscalecollaboration services deal with such collaboration issuesas cross-scale information sharing and cross-scale avatarmanipulation These services lay the foundation for multi-scale tools

43 Implementation of Multiscale Tools

A set of multiscale collaboration tools were imple-mented and integrated into the prototype system Thissection introduces such tools as scaling control whichdistinguishes mCVEs from other CVEs scale-based se-mantic representations which visualize information in amultiscale way cross-scale dynamic view transitionwhich addresses the cross-scale information problemuniquely seen in multiscale collaboration and the man-agement of avatar position in direct avatar manipulationwhich illustrates cross-scale data coordination

431 Scene Graph of the Prototype Systemand Scaling Control The scene graph of the systemis shown in Figure 10 Each Java 3D scene graph has aVirtual Universe object as the root A Locale object at-taches scene graphs to the Virtual Universe objectBranchGroup (BG) nodes collect such objects as geom-etry behavior light sound and so forth A Transform-Group (TG) node defines the coordinates of objectsunder it

The system has two subscene graphs the object scenegraph which is the left child of the Locale object andthe view scene graph which is the right child The ob-ject scene graph collects all objects and behavior nodesThe view scene graph defines the viewing behaviors

To change a userrsquos interaction scale is to change thevalues of the userrsquos interaction parameters relative to thesize of the virtual world Thus scale change can be im-plemented in two ways (1) scaling the virtual world upand down and keeping all interaction parameters un-

touched or (2) changing all interaction parameters andkeeping the virtual world untouched Mathematicallythey are equivalent We adopted the first approach be-cause scaling the virtual world up and down can bedone by modifying just one parameter the scaling fac-tor Si of the top TG in the object scene graph This ismuch simpler than the second approach which involvesthe modification of complicated view parameters

432 Scale-Based Semantic RepresentationsIn the implementation of scale-based semantic represen-tations we adopted a design that uses a delegate objectto wrap all representations and that delivers appropriaterepresentations based on the value of the controllingvariable (Fox 1998) A scale-based semantic representa-tion node is extended from the Java 3D Switch objectEach node has a set of child representations and an or-dered array that defines the scale range for each repre-sentation (Figure 11)

433 Animating View Transition in Collabo-rative View Sharing The key issue in making a view-transition animation is to interpolate two views InmCVEs a view can be uniquely determined by its viewposition P its view orientation O and its view scale Sand be written as V(P O S)

Given two views V0 V(P0 O 0 S0) and V1 V(P1O 1 S1) to implement a view-transition animation is to

Figure 10 Scene graph of the prototype system

Zhang and Furnas 41

choose a path that links V0 and V1 and to set the trajec-tory of viewpoint movement view direction along thepath and observation scale along the path View-pathand view-orientation interpolation techniques are ma-ture (Parent 2001 Watt amp Policarpo 2001) so thisresearch focused only on the interpolation of view scale

A scale-interpolation function could be linear loga-rithmic or in other forms We chose a logarithmic func-tion because it gives users a constant relative rate ofchange in their views Interpolating the view scale loga-rithmically can give users a familiar experience

Assume the animation includes n frames from V0 toV1 The view scale of the ith frame (0 i n) in a log-arithmic interpolation function can be written as

Si Si S0 (1)

where

S S1

S01n

(2)

With corresponding P i and O i from interpolationfunctions for view position and view orientation theview of the ith frame Vi V(P i O i Si) can be ob-tained Figure 12 shows two views seen in Figure 7 asFrame 0 and Frame n and other interpolated viewframes (Due to space limitations we have shown onlyfour frames used for the animation) Showing theseframes successively with a reasonable frame rate can pro-duce an animation linking these two divergent views

434 Cross-Scale Data Consistency and Ava-tar Position in Direct Avatar Manipulation Al-though changing the scaling factor of the top TG in theobject scene graph simplifies the scaling operation such

an approach poses a challenge in cross-scale data consis-tency In CVEs all users usually need to see the sameobject appearing at the same spatial location In conven-tional CVEs this can be easily achieved because an ob-jectrsquos coordinates in different usersrsquo worlds are often thesame In mCVEs however when users are at differentscales their virtual worlds are rescaled according to thescaling factors of the top TG in their own object scenegraphs The same object will have different local coordi-nates in different usersrsquo worlds When the object ismoved by a user the objectrsquos local coordinates in thisuserrsquos world cannot be used to update the objectrsquos newposition in all other usersrsquo worlds

In the implementation we used normalized coordi-nates which are calculated as the ratio of the object lo-cal coordinates in the userrsquos world to the userrsquos scaleLet Pl(xl yl zl) and Sl denote the local coordinates andthe userrsquos scale respectively The normalized coordi-nates Pn can be written as

Pn PlSl (3)

By doing so the same object will always have the samePn This Pn is used to define the coordinates of the ob-ject in every userrsquos object scene graph Modifying thespatial location of an object in a userrsquos world leads to anew Pn All users can use this Pn to update the coordi-nates of the object in their object scene graphs There-fore Pn enforces data consistency across scale Becausethe scaling factor is on the top of the scene graph theobjectrsquos final location in a userrsquos world is Pl the productof Pn and this userrsquos scale Sl With different Sl users willbe able to see the world rendered at different scales

An avatarrsquos position can be managed similarly butwith a small difference An avatar object differs fromother ordinary objects in a way that in its host userrsquos

Figure 11 Scale-based semantic representation node

Figure 12 Frames of a dynamic view-transition animation

42 PRESENCE VOLUME 14 NUMBER 1

world the coordinates of the avatar are affiliated withthe view scene graph rather than the object scenegraph so that the avatar always moves with the view Inother usersrsquo worlds however the avatar object is underthe object scene graphs Figure 13(a) is the scene graphof the world of a user say A who interacts with the vir-tual environment at a location Pa and with a scale SaFigure 13(b) is the scene graph of the world of anotheruser say B User Arsquos avatar is in User Brsquos object graphBecause Pa is not governed by Sa the normalized coor-dinates of the avatar in User Brsquos object graph should bewritten as

PnPaSa (4)

The body size of User Arsquos avatar changes with thescale of User A so a scale parameter is also needed todefine the size of User Arsquos avatar in User Brsquos objectscene graph so that User B can be aware of User Arsquosscale This scale should also be normalized Let Sn de-note the normalized scale where Sn is simply the recip-rocal of Sa

Sn 1Sa (5)

This is because the avatar body changes inversely pro-portional to the scale factor Sa (shrinking the world isequivalent to growing the avatar body and growing theworld is equivalent to shrinking the avatar body)

The implementation of other cross-scale tools includ-ing direct avatar manipulation also uses Pn and Sn todefine the coordinates and size of an avatar entity Thisapproach guarantees the consistency of the avatar posi-tion and size in all usersrsquo views at different scales

5 Experimental Evaluation

We evaluated multiscale collaboration in subjecttests An experiment was designed to assess the benefitsusers can get from an mCVE in accomplishing tasks re-quiring cross-scale coordination

51 Experiment Design and Procedure

Subject tasks in the experiment involved searchingfor a ldquobombrdquo on a square ground plane (2000 2000m2) with a distinctly shaped building in each corner(square hexagon octagon and circle as seen fromabove) Each had a height of 12 m and a base of about6400 m2 On the ground behind each building was aunit cube (1 m3) containing a unique text name andsmiley face (about 05 05 m2) The bomb was insideone of these four cubes In the test subjects were placedin the middle of the square plane They had a defaulteye level of 168 m and a default moving step of 1 m inthe virtual environment The shape of the building thatthe bomb was nearby was known in advance and sub-jects needed to find that building locate the bomb boxand key in the name of the box to defuse the bomb

A 2 2 2 factorial design that has six treatmentstwo noncollaboration and four collaboration wasadopted (Table 1) For the two noncollaboration treat-ments VE is just a conventional 3D virtual environ-ment and M-VE is a VE enhanced by multiscale toolsspecifically it allows users to change their interactionscales eye level and speed Among four collaborationtreatments CVE a conventional collaborative virtualenvironment is the only one without multiscale toolsThe other three treatments all equipped with multiscale

Figure 13 Two scene graphs for the same avatar

Table 1 Six Treatments in a 2 2 2 Design

Noncollaboration Collaboration

Nonmultiscale VE CVEMultiscale M-VE NR

GUIDEMOVE

Zhang and Furnas 43

tools differ in the assignment of subject task roles (be-ing a giant or a normal person) and in the way subjectsaffect each otherrsquos work across scales In one treatmentthe roles of subjects are not predefined and subjects canchoose their own interaction scales as desired This envi-ronment is labeled as NR (No Role) The other twomCVEs both assign one subject to be a giant and theother a normal person so that subjects can change theirinteraction scales only within a limited range In onesuch environment the giant is permitted to move thepartner directly and this treatment is labeled as MOVEAnother condition allows the giant to guide the move-ment of the partner only verbally and is denoted asGUIDE

Recruited through email 24 students paired in 12groups participated in the experiment In noncollabora-tion treatments they worked on their own and in col-laboration treatments they communicated through anaudio channel The performances were measured bytask-completion time

52 Results

An ANOVA analysis of data from four treatments(VE M-VE CVE and MOVE) shows main effects ofboth collaboration (F170 1298 p 0001) andmultiscale (F170 7090 p 00001) Interaction isnot significant (F170 187 p 176) as seen in Fig-ure 14 Subjects performed best in MOVE where theycould take full advantage of multiscale and collabora-tion and they did worst in VE where there was no helpat all

Subjects used different strategies in the different con-ditions Without multiscale tools subjects had to goaround and count the number of sides of all four build-ings to find the target a very time-consuming processWith multiscale subjects could increase their sizes to seebuilding shapes from above and approach the bombquickly so the time required was be reduced signifi-cantly

The performance difference among the three mCVEsalso indicates the importance of different collaborationsupports In the NR treatment subjects had to assumetheir roles via an expensive negotiation process In the

GUIDE treatment the giant subject who could see thebuilding shape must coordinate with the partner aboutnavigation verbally through a costly grounding process(Clark amp Brennan 1991) In MOVE however the all-seeing giant could actually move the partner to the des-tination quickly and bypassed grounding processes

6 Discussion

In this paper we have introduced a novel interac-tive environment multiscale collaborative virtual envi-ronment which gives users the freedom to control theirinteraction scales in collaboration Multiscale collabora-tion can help better allocate scarce scale resources andcombine different expertise and action capabilities ofindividual users in their work with large structures Wehave also discussed the implications of multiscale tech-nology for user interaction and the design and develop-ment of a Java-based mCVE system Our subject testshave shown that multiscale collaboration can facilitatepeoplersquos work that requires cross-scale information andcollaboration

Research reported in this paper has largely focused ongeneral interaction issues in mCVEs Designs have beenlimited to generic tools to support multiscale collabora-tion and experimental study has focused on generaltasks We expect that multiscale collaboration tools canbe used to support peoplersquos specific work in the realworld To achieve this goal the design of multiscale

Figure 14 Time (in seconds) for the six treatments

44 PRESENCE VOLUME 14 NUMBER 1

tools should support domain-specific tasks One of ourongoing research projects focuses on studying how multi-scale collaboration can support research in materials sci-ence and engineering We are investigating the needs ofmaterials scientists for multiscale collaboration and willdesign and implement tools that support these needsThe effectiveness and usability of these tools will also beevaluated when they are ready

At the same time we are interested in behavior issuesin mCVEs Some important research questions concern-ing user behavior in mCVEs have not been studiedthoroughly It is still unclear when and how users canbenefit from collaborative object manipulation acrossscale whether the designed dynamic view-transitiontechnique can indeed improve cross-scale informationsharing what mechanism is needed to help users bettermanage cross-scale collaboration overheads and soforth In addition it has been found that multiscale alsobrings some new challenges in social interactions(Zhang amp Furnas 2002) and efforts are needed tostudy what can be done to address these interaction is-sues Research in this direction will deepen our under-standing of mCVEs and help to design better tools tosupport multiscale collaboration

Acknowledgments

This research was funded in part by Microsoft Research

References

Ahl V amp Allen T (1996) Hierarchy theory A vision vocab-ulary and epistemology New York Columbia UniversityPress

Alexander C Ishikawa S amp Silverstein M (1977) A pat-tern language Oxford UK Oxford University Press

Bederson B amp Hollan J (1994) Pad A zoominggraphical interface for exploring alternate interface physicsProceedings of ACM UIST rsquo94 17ndash26

Benford S Bowers J Fahlen L Greenhalgh C amp Snow-don D (1995) User embodiment in collaborative virtualenvironments Proceedings of ACM CHI rsquo95 242ndash249

Benford S Snowdon D Colebourne A OrsquoBrien J ampRodden T (1997) Informing the design of collaborativevirtual environments Proceedings of ACM GROUP rsquo97 71ndash80

Carlsson C amp Hagsand O (1993) DIVEmdashA platform formulti-user virtual environments Computer and Graphics17(6) 663ndash669

Clark H amp Brennan S (1991) Grounding in communica-tion In L B Resnick J Levine amp S D Behrend (Eds)Perspectives on socially shared cognition (pp 127ndash149)Washington DC American Psychological Association

Dix A J (1994) Computer-supported cooperative workmdashAframework In D Rosenburg amp C Hutchison (Eds) De-sign Issues in CSCW (pp 23ndash37) Berlin Springer-Verlag

EPFL Geneva IIS TNO Lancaster UCL et al (1997)Guidelines for building CVE applications COVEN ProjectRetrieved November 21 2002 from httpwwwcsuclacukstaffmslatervrCovenPublicDeliverablesdel26PDF

Fox D (1998) Tabula rasa A multi-scale user interface sys-tem Unpublished doctoral dissertation New York Univer-sity

Furnas G (1986) Generalized fisheye views Proceedings ofACM CHI rsquo86 16ndash23

Gaver W Sellen A Heath C amp Luff P (1993) One isnot enough Multiple views in a media space Proceedings ofACM INTERCHI rsquo93 335ndash341

Hollan J amp Stornetta S (1992) Beyond being there Pro-ceedings of ACM CHI rsquo92 119ndash125

Koike H amp Yoshihara H (1993) Fractal approaches forvisualizing huge hierarchies Proceedings of IEEE VL rsquo9355ndash60

Lamping J Rao R amp Pirolli P (1995) A focus contexttechnique based on hyperbolic geometry for visualizinglarge hierarchies Proceedings of ACM CHI rsquo95 401ndash408

Leigh J Johnson A amp DeFanti T (1996) CALVIN Animmersimedia design environment utilizing heterogeneousperspectives Proceedings of IEEE International Conferenceon Multimedia Computing and Systems 20ndash23

Lieberman H (1994) Powers of ten thousand Navigating inlarge information spaces Proceedings of ACM UIST rsquo9415ndash16

Mackinlay J Card S amp Robertson G (1990) Rapid con-trolled movement through a virtual 3D workspace Proceed-ings of ACM SIGGRAPH rsquo90 171ndash176

Mine M Brooks F amp Sequin C (1997) Moving objects

Zhang and Furnas 45

in space Exploiting proprioception in virtual-environmentinteraction Proceedings of ACM SIGGRAPH rsquo97 19ndash26

Morrison P amp Morrison P (1994) Powers of ten About therelative size of things in the universe and the effect of addinganother zero San Francisco W H Freeman

Parent R (2001) Computer animation Algorithms and tech-niques San Francisco Morgan-Kaufmann

Pattee H (1973) Hierarchy theory The challenge of complexsystems New York George Braziller

Perlin K amp Fox D (1993) Pad An alternative approach tothe computer interface Proceedings of ACM SIGGRAPHrsquo93 57ndash64

Pierce J Forsberg A Conway M Hong S Zeleznik Ramp Mine M (1997) Image plane interaction techniques in3D immersive environments Proceedings of the 1997 Sympo-sium on Interactive 3D Graphics 39ndash43

Poupyrev I Billinghurst M Weghorst S amp Ichikawa T(1996) The go-go interaction technique Non-linear map-ping for direct manipulation in VR Proceedings of ACMUIST rsquo96 79ndash80

Puppo E amp Scopigno R (1997) Simplification LOD multi-resolution Principles and applications Proceedings ofEUROGRAPHICS 16 (3)

Raab A amp Ruger M (1996) 3D-Zoom Interactive visual-ization of structures and relations in complex graphics InB Girod H Nieman amp H Seidel (Eds) 3D image analy-sis and synthesis (pp 125ndash132) Sankt Augustin GermanyInfix-Verlag

Robbins M (2001) Modeling and simulation of the failureand toughness of polymersubstrate bonds The second An-nual Multiscale ModelingmdashBridging the Gap Between Atom-istic and Meso Scales

Robertson G Card S amp Mackinlay J (1991) Informationvisualization using 3D interactive animation Proceedings ofACM CHI rsquo91 461ndash462

Robertson G amp Mackinlay J (1993) The document lensProceedings of ACM UIST rsquo93 101ndash110

Robertson G Mackinlay J amp Card S (1991) Cone treesAnimated 3D visualizations of hierarchical informationCommunications of the ACM 34 (2) p 89ndash194

Sarkar M amp Brown M H (1992) Graphical fisheye viewsof graphs Proceedings of ACM CHI rsquo92 83ndash91

Singhal S amp Zyda M (1999) Networked virtual environ-ments Design and implementation New York ACM Press

Snowdon D amp Jaa-Aro K (1997) A subjective virtual envi-ronment for collaborative information visualization VirtualReality Universe rsquo97 Retrieved May 23 2003 fromhttpwwwcyberedgecomvru_paperssnowdenhtm

Stappers P Gaver W amp Overbeeke C (2000) Beyond thelimits of real-time realism 2000 Chapter to appear in L JHettinger amp M W Haas (Eds) Psychological issues in thedesign and use of virtual and adaptive environment Mah-wah NJ Erlbaum

Stefik M Bobrow D Foster G Lanning S amp Tatar D(1987) WYSIWIS revised Early experiences with multiuserinterfaces ACM Transactions on Information Systems(TOIS) 5(2) 147ndash167

Stoakley R Conway M amp Pausch R (1995) Virtual real-ity on a WIM Interactive worlds in miniature Proceedingsof ACM CHI rsquo95 265ndash272

Sutherland I (1965) The ultimate display Proceedings ofIFIP Congress 2

Tromp J G (1999) Usability design for CVEs COVENProject Retrieved November 21 2002 from httpcovenlancsacuk4deliverablesdel29pdf

Waters R Anderson D Barrus J Brogan D Casey MMcKeown S et al (1997) Diamond park and SPLINESocial virtual reality with 3D animation spoken interactionand runtime extendibility Presence Teleoperators and Vir-tual Environments 6(4) 461ndash481

Watt A amp Policarpo F (2001) 3D games Animation andadvanced real-time rendering New York Addison-Wesley

Zhang X amp Furnas G (2002) Social interactions in multi-scale CVEs Proceedings of ACM CVE 2002 31ndash38

46 PRESENCE VOLUME 14 NUMBER 1

Gaver amp Overbeeke 2000) we can expand designspace and find new ways to address old problems Thissection discusses the features of multiscale virtual envi-ronments (mVEs) the necessity of multiscale collabora-tion and the benefits of mCVEs Relevant literature isalso reviewed

21 Multiscale Virtual Environments

The absence of real-world physics in virtual envi-ronments has led to some innovative designs in virtualenvironments Teleportation for example is a usefulnavigation tool that transcends speed and time AnmVE is a design that overcomes the scale barrier andallows users to manipulate the scale of the virtual spacein their work

211 Scale in Virtual Environments In com-puter graphics scale usually refers to the rendered sizeof objects To manipulate the scale of a virtual environ-ment is to resize the virtual environment Figure 4shows images of a DNA structure at two scales Thesetwo images differ only in rendered sizes and resultingdifferent levels of detail However if the scale differenceis more dramatic image difference can go beyond geo-metric size as seen in Figure 5 where three images atsignificantly different scales show very different and im-portant characteristics the double helix in (a) the mo-lecular structure in (b) and the nucleus and the electroncloud in an atom in (c)

212 Scale in Multiscale Virtual Environ-ments This scale-dependent rendering technique dis-tinguishes mVEs from conventional virtual environ-ments Conventional virtual environments renderobjects with the same representation regardless of userinterests mVEs render objects with different representa-tions corresponding to different scale ranges In mVEsusers can easily change their observation scales and re-ceive different representations from big pictures to localdetails

Another important but less obvious characteristic ofmVEs is that users also get different interaction do-mains To change the scale of a virtual environment isto shift a userrsquos action capabilities One action can leadto different results at different scales For example auserrsquos movement could be at the nanometer level whenthe user is at the atomic scale and the same movementwould be at the micrometer level when the user movesto the molecular scale When the user observes a virtualplanet system the movement scale can be at the level ofhundreds of thousands of kilometers In mVEs userobservation and action capabilities are coordinated atthe same or comparable scale level

Users need this association in mVEs so that they canwork appropriately with what they see For examplewhen users observe an object structure at the level of 1mm their movement scale should be at a comparablescale say 1 mmsec Moving at a very different scalesay 1 kmsec would make it hard to keep a consistentview Objects of interest would be constantly speedingout of sight Conversely if the structure of interest is atthe level of 1 km moving at a scale of 1 mmsec would

Figure 4 DNA rendered at two different scales

Figure 5 Successive views of a DNA structure rendered at different

scales Perspective lines added for the figure to show where each

successive view comes from

34 PRESENCE VOLUME 14 NUMBER 1

be problematic It would demand tremendous effort tochange the view of the large structure even slightlyOther interaction parameters such as reachable distanceand manipulation accuracy should be similarly coordi-nated

Thus mVEs provide users with new interaction capa-bilities that are usually unavailable in conventional vir-tual environments In mVEs users can interact withvirtual worlds at multiple scale levels At any given scalesome things will be easy to see and manipulate whileothers will be either too large or too small to displayChanging the scale moves new things into the easilyvisible and manipulative range Coordinated observationand action capabilities help users to better interact withmultiscale structures Multiscale extends the designspace for information visualization and user interaction

22 Multiscale Collaboration

Multiscale collaboration can further facilitate userinteraction with multiscale structures In particular itcan help people with work on large structures that needmore labor on rapidly changing structures that demandparallel working and on complex structures that requiredomain knowledge beyond what an individual mayhave

221 Large Multiscale Structures Managinga large multiscale structure could be difficult for a singleuser Take a city-planning example Adding new designelements (eg paths or landmarks) may force plannersto move back and forth frequently between differentscales and check the impact of new additions on designat different levels For a large city continuously chang-ing scale could be very demanding and costly if there isjust one planner Multiscale collaboration can offersome help by placing individual planners with differentscales and having them focus on issues at their own scalelevel Planners can then work independently on prob-lems within their own scale scopes and collaborativelydeal with cross-scale issues Collaboration divides bigcross-scale problems into smaller within-scale ones thatcan be more easily handled by individuals

222 Rapidly Changing Multiscale StructuresIt could also be a challenge for a user to manage a struc-ture that changes rapidly however small that structuremay be Consider a scenario in which a small computernetwork managed by a system administrator is beinginfected by a worm virus The system administrator maynot be able to keep pace with the speed at which thevirus spreads If more system administrators are in-volved they can have individuals to check subnets atdifferent levels simultaneously and then take appropri-ate actions to stop the virus from further damaging theentire network Collaboration by supporting parallelworking improves peoplersquos response to problems atdifferent levels

223 Cross-Domain Multiscale StructuresChallenges may also come from the diversified knowl-edge and expertise required in understanding and man-aging structures spanning across the boundaries of sci-entific domains Understanding cross-domain multiscalestructures could be daunting for individuals For exam-ple in studying the strength of a new material acrossdifferent scale levels no single person would possess allthe required in-depth knowledge of chemistry materialsscience and mechanical engineering Chemists materi-als scientists and mechanical engineers may need towork together on this issue While they can focus onproblems in their own expertise domains individuallyworking together helps them see how properties at dif-ferent scale levels may affect each other and even dis-cover important issues that might be ignored withoutcross-scale collaboration Collaboration helps solvecross-scale problems by combining peoplersquos knowledgedomains from different scales

23 Multiscale Collaborative VirtualEnvironments

An mCVE is an mVE that supports collaborationIn mCVEs users can choose their own scales in the ob-servation of an object and work together on the sameobject at different scales A team of city planners forexample can have a regular planner of the normal hu-man size at street level and a ldquogiantrdquo planner at the city

Zhang and Furnas 35

level While each planner focuses on planning issues athis or her own level they can help each other on cross-scale issues The giant planner can take the street-levelplanner quickly to a distant destination and the street-level planner can help the partner place buildings orlandmarks at their exact locations Figure 6 shows twoplanners at two scales working together Their cross-scale collaboration could help make a plan that meetsthe requirements at various levels

As seen in Figure 6 each user is represented as anavatar in mCVEs with a scaled size corresponding tothe scale level of the user This avatar metaphor offerstwo benefits First avatars provide rich awareness infor-mation (eg user view locations viewing directionsbody gestures) that are critical to social interactions invirtual environments (Benford Bowers Fahlen Green-halgh amp Snowdon 1995) Second by yoking differentobservation and action parameters under the same ava-tar body avatars can help users better understand theassociation of different observation and action capabili-ties in mCVEs The eye level of a userrsquos avatar tells howfar the user can see the avatarrsquos arm length indicateshow far the user can reach the leg length shows howfast the user can move Seeing the size of a userrsquos avatarother users can tell at what scale the user is working andwhat the user can do

mCVEs could be a promising tool to help people dealwith multiscale structures but this new design paradigmraises many research issues ranging from the conceptualunderstanding of the implications of multiscale for userperception and action to the technical design of

mCVEs to the experimental evaluation of the effective-ness of mCVEs

24 Literature Review

Some research effort has been made to addressdesign issues related to user interaction with large struc-tures Furnas (1986) proposed a theoretical frameworkGeneralized Fisheye Views to manage large hierarchicalstructures by allocating more space for objects of inter-est This technique has been used to visualize varioustypes of data (Sarkar amp Brown 1992 Robertson ampMackinlay 1993 Lamping Rao amp Pirolli 1995 Raabamp Ruger 1996) Koike and Yoshihara (1993) used thefractal concept to expand the tree structure based on thedegrees of interest Cone Trees were built to present filehierarchies (Robertson Mackinlay amp Card 1991)These methods focused only on better using screenspace to deal with the size of information structures butgave little consideration to visualizing structures withdifferent representations based on user interactionscales Multiscale technology (Perlin amp Fox 1993 Bed-erson amp Hollan 1994 Lieberman 1994) providedearly inspiration for this research This technique wasusually regarded as an alternative user interface in 2DThis research extends it into 3D worlds and argues thatmultiscale technology is a powerful design that by al-lowing people to control their observation and actiondomains at different scales can augment our limitedcognitive capabilities

Scaling has been seen in some designs to support useractions in virtual environments Scaling tools were usedto increase reaching distance (Poupyrev BillinghurstWeghorst amp Ichikawa 1996 Mine Brooks amp Sequin1997 Pierce et al 1997) facilitate spatial knowledgeacquisition (Stoakley Conway amp Pausch 1995) andimprove navigation performances (Mackinlay Card ampRobertson 1990) These scaling tools usually focusedon only one interaction parameter and ignored the im-plications of the change of one parameter for all otherrelated perception and action While isolating individualinteraction parameters may work well in some situa-tions uncoordinated perception and action as discussed

Figure 6 Two city planners working at two different scales

36 PRESENCE VOLUME 14 NUMBER 1

previously could cause some problems in the interac-tion with large multiscale structures

Research literature on virtual environments support-ing collaboration is massive but consideration of cross-scale collaboration is rarely seen The most relevantproject is the CALVIN system (Leigh Johnson amp De-Fanti 1996) in which two users can collaborate on adesign project from two different perspectives In thesetwo perspectives objects were rendered with same rep-resentations and the system did not provide users withmultiple levels of abstraction at different scales Alsothe system gave users limited choices of working scalesand could not satisfy their needs for active and interac-tive control over scale across a much broader range

3 Design of mCVEs

There are some design guidelines for 3D collabo-rative virtual environments (CVEs) (Benford SnowdonColebourne OrsquoBrien amp Rodden 1997 EPFL et al1997 Tromp 1999) However they were constructedfor the design of conventional CVEs and common col-laboration actions in CVEs Applying them in designingmCVEs could be difficult because scale issues have notbeen considered in these guidelines In this research weadopt a design approach that starts from understandingthe implications of multiscale technology for user inter-actions and then derives design considerations based onthis understanding In this section efforts are made firstto analyze how multiscale may affect user interactionThen some conceptual designs are proposed to supportuser perception and action in mCVEs

31 Implications of Multiscale for UserInteraction

Multiscale could affect user interactions at differ-ent levels and in different ways Here we first discusshow multiscale can support the understanding of multi-scale structures Then we describe how it may impedeinformation sharing among users at different scales andhow it may facilitate cross-scale action

311 Understanding Multiscale StructuresAs discussed previously interacting with large structuresrequires users to obtain both detailed content and suffi-cient context information This well-known ldquocontent

contextrdquo problem is indeed a multiscale problem be-cause detailed content information and global contextinformation are usually distributed at different scale lev-els To understand complicated structures that presentdifferent features at different scales we usually breakphenomena into components and focus our attentionson issues within certain levels rather than all levels atonce (Pattee 1973 Ahl amp Allen 1996) and thenchoose appropriate tools (eg microscopes telescopes)for observation and analysis at different scales

In virtual environments multiscale technology couldhelp address this ldquofocus contextrdquo problem by allowingusers to control what they want to see and what they cansee from multiscale structures and then to manipulate theamount of context and content information displayed onthe screen Under different interaction scales objects canbe rendered at different sizes tiny objects can be madelarger for more detailed observation and huge objects canbe made smaller for a better big picture The same struc-ture can have different appearances at different scales in-forming users with the rich features of the structure asseen in Figure 5 The understanding of a complicatedstructure could be improved

312 Cross-Scale Information Sharing Col-laboration can facilitate peoplersquos work by improvingcommunications andor cooperation on shared artifacts(Dix 1994) Multiscale collaboration focuses more onartifacts than communications In such artifact-centeredcollaboration users often need to refer to shared arti-facts (Dix) Conventional CVEs are usually ldquoWhat-You-See-Is-What-I-Seerdquo (WYSIWIS) or ldquorelaxed WYSIWISrdquo(Stefik Bobrow Foster Lanning amp Tatar 1987) soknowing what others are referring to is fairly easymCVEs are not like this however Users may have verydifferent views at different scales and see totally differ-ent objects and structures Figure 7 shows the view of acity planner who oversees the whole city (a) and theview of another planner who checks buildings at thestreet level (b) These two views which are from very

Zhang and Furnas 37

different perspectives and show different levels of ab-straction of the city may make it difficult for two plan-ners to understand how their work can be related Aplanner can refer to a particular object in her view butthe other may not see it at all Multiscale technologymay hurt mutual understanding of objects in collabora-tion because of scale-related views

313 Cross-Scale Action Multiscale can alsobring some benefits to collaboration One of them isthat users can leverage their different action capabilitiesand different scopes of influence of their actions Takethe example of the two city planners again When thestreet-level planner needs to go to a distant place thecity-level planner with a great action domain can movethe partner easily to the destination and reduce the tasktime Or when the city-level planner wants to move toan exact location the street-level planner can move thepartnerrsquos avatar accurately because of high manipulationaccuracy It is mentioned previously that a similar tech-nique can be used in object manipulation Thus multi-scale collaboration can improve navigation and manipu-lation activities by allowing users to take advantage ofeach otherrsquos different action capabilities

32 Design Considerations

Since multiscale technology could affect user in-teraction both positively and negatively the design ofmCVEs focuses on maximizing the advantages of multi-scale technology and avoiding the disadvantages Thisprinciple leads to the following design considerations

321 Support for Better Understanding ofMultiscale Structures Multiscale structures shouldbe modeled in such a way that scale-dependent charac-teristics can be observed at different scale levels In thereal world multiscale structures are ubiquitous as seenin Figures 1 and 2 To observe these multiscale struc-tures we need only appropriate tools In 3D virtualworlds however what users can see is predesignedThus to visualize multiscale structures objects have tobe modeled and rendered in a multiscale way

322 Support for Cross-Scale InformationSharing Users should be able to share artifacts ofcommon interest with others across scale Tools such asmultiple views of the world (Gaver Sellen Heath ampLuff 1993) cannot help too much Such techniquesrequire some common objects in different views as ref-erences but usersrsquo very subjective views in mCVEs maymake it difficult to find such references It is a challengeto build mutual understanding of working contexts withsubjective views (Snowdon amp Jaa-Aro 1997) Usersneed help in understanding how their subjective viewsand objects in them are related to each other

323 Support for Cross-Scale Action Usersshould be allowed to get involved in othersrsquo work acrossscale so that they can leverage their different interactioncapabilities in collaboration They should be able to di-rectly manipulate objects that others are working onand move the avatars of other users Of course this in-tervention should be based on mutual agreements toavoid any unwanted consequences In the city-planningexample the city-level planner should not change thingsthe street-level planner is working on without consentOtherwise the street-level planner may get confusedand not understand what has happened

33 Conceptual Design

Based on these considerations some multiscaletools were designed Here discussions focus on designsthat are fundamentally important to mCVEs includingscale-based semantic representations that present scale-related characteristics of multiscale structures dynamic

Figure 7 Two views of a city at two scale levels

38 PRESENCE VOLUME 14 NUMBER 1

view transition that facilitates cross-scale informationsharing and direct avatar manipulation that supportscross-scale action

331 Scale-Based Semantic Representationsfor Multiscale Structures This design renders amultiscale structure with different representations atdifferent scales Figure 5 presents the visual results of asubstance under this technique Initially the substanceappears as a double helix in (a) The user changes thescale to increase the structure size and to see more de-tails in (b) Continuing the scaling the user finds a dif-ferent structure in an atom in (c) The user is informedof semantic characteristics of the substance at differentscales

This scale-based representation technique has oneproblem The change between two very different repre-sentations may confuse users For example the suddenappearance of the structure in Figure 5(c) may distractusers from their primary tasks To address this issue weadopted a fading tool (Bederson amp Hollan 1994Lieberman 1994) to smooth the transition betweendifferent representations Figure 8 shows a transitionprocess from a spherelike molecule from Figure 8(a) toa set of atoms inside it in Figure 8(d) Instead of jump-ing abruptly between different representations the dis-appearance of the molecule and the appearance of atomsare gradual The transition between two representationsis more easily comprehensible Cognition required inunderstanding the change of visual image can be kept atthe level of perceptual processing so that users can freetheir cognitive processing capacities for more compli-cated tasks (Robertson Card amp Mackinlay 1991)

Scale-based semantic representations can be seen asone member of the family of level of detail (LOD) tech-

niques which visualize the same structure with differentgeometric objects according to a particular interactionparameter but with a very different purpose Whilemost LOD techniques such as distance LOD primarilyconcern computation efficiency (Puppo amp Scopigno1997) scale-based semantic representation gives moreconsideration to the userrsquos interaction needs CommonLOD techniques render individual objects with differenttextures or geometries but scale-based semantic repre-sentation visualizes structures with different semanticabstractions LOD techniques can be integrated intoscale-based semantic representations to improve bothuser interaction and machine performance For examplein a city-planning project a city can have different struc-tures at the level of city district and street to informplanners semantically and distance LOD can be appliedin rendering individual objects in each structure to im-prove 3D computation

332 Dynamic View to Support Cross-ScaleInformation Sharing Design here addresses issuesconcerning how to help users at different scales to buildcommon understandings of their divergent views Onedesign choice is to use a dynamic view to narrow downthe view difference We used animation to show thetransition between two views Animation is generatedby interpolating the view positions view orientationsand view scales of two views For example if two usersrsquoviews at two different scales look like Figures 7(a) and7(b) an animation can be generated by adding interme-diate image frames between Figure 7(a) and 7(b) Theanimation can tell how one view can be transformed toanother and how objects in two views are related toeach other

333 Direct Avatar Manipulation to SupportCross-Scale Action To help users get involved inothersrsquo work so that they can take advantage of theirdifferent action capabilities we adopted a design choicethat allows direct manipulation on other usersrsquo avatarsWith such a tool planners in the above city-planningexample can easily cooperate on tasks that require largeaction domain and high action accuracy Of course thisdirect interference should be regulated The user who

Figure 8 Successive fading views in semantic representations

Zhang and Furnas 39

owns the avatar is always aware of such action and hascontrol over whether this action is allowed

The implication of such a direct manipulation of ava-tar body for user interface design deserves more atten-tion Under this design an avatar can ldquoaffordrdquo manipu-lation activities This is quite different from thetraditional role of an avatar merely as user embodimentin conventional CVEs The reason that no direct manip-ulation of an avatar is allowed in conventional CVEsmight be due to the influence of real life where the au-tonomy of human bodies is well respected To betterfacilitate peoplersquos work in virtual environments somereal-life constraints could be relaxed or even abandonedVirtual environments should not be treated as the repli-cation of the real world just as new technologies shouldnot be interpreted by old metaphors Direct manipula-tion of a userrsquos avatar by others is an effort to breakdown the metaphor of reality

4 System Implementation

A desktop mCVE prototype system consisting ofover 20000 lines of Java code was implemented Dis-cussions in this section focus on the choice of toolkits insoftware implementation general system architectureand the implementation of some multiscale tools

41 Choice of Toolkits

The design of virtual environments that supportcollaboration needs to consider issues in three aspects3D graphics rendering network communication man-agement and user interactions (Singhal amp Zyda 1999)It is not our interest to address issues related to low-level graphics rendering or network communicationsOur goal here is to support user interactions with multi-scale structures and with other users at different scalesTo achieve this interaction-oriented design goal weimplemented the mCVE prototype with two toolkitsJava 3D and Java Shared Data Toolkit (JSDT)

This hybrid approach allows a design focus on high-level interaction tools because Java 3D can handlegraphics rendering and JSDT can handle network com-

munications issues With the scene-graph managementof Java 3D we can construct virtual environments andstructures at the level of geometric objects rather thanat the level of vertices and polygons Java 3D allows theextension of new rendering behaviors which is not wellsupported by other CVE toolkits such as DIVE (Carls-son amp Hagsand 1993) and SPLINE (Waters et al1997) This flexibility is critical to the implementationof such tools as scale-based semantic representations anddynamic view transition Furthermore the easy integra-tion of other Java APIs such as Java Swing into Java3D can simplify user interface design JSDT is ideal forthe research because it provides flexible data deliveryservices in support of synchronous collaboration on theInternet JSDT supports basic abstraction of networksessions multicast message communications and con-currency control Thus the design can focus on opti-mizing the management of distributed 3D scenes andcross-scale communications One drawback of this hy-brid approach is the lack of those CVE services seen inSPLINE (Waters et al) Collaboration services have tobe built from scratch

42 System Layer and Architecture

The system layer of the prototype is shown in Fig-ure 9 The implementation focused on the two layersinside the dashed rectangle On top of Java 3D andJSDT are multiscale graphics services and collaboration

Figure 9 System layer of the mCVE prototype

40 PRESENCE VOLUME 14 NUMBER 1

services and above them are multiscale collaborationtools

Multiscale graphics services handle object-renderingissues and update user views based on their view posi-tions and interaction scales Collaboration services de-liver messages among distributed users and manage theconsistency of shared virtual environments Multiscalecollaboration services deal with such collaboration issuesas cross-scale information sharing and cross-scale avatarmanipulation These services lay the foundation for multi-scale tools

43 Implementation of Multiscale Tools

A set of multiscale collaboration tools were imple-mented and integrated into the prototype system Thissection introduces such tools as scaling control whichdistinguishes mCVEs from other CVEs scale-based se-mantic representations which visualize information in amultiscale way cross-scale dynamic view transitionwhich addresses the cross-scale information problemuniquely seen in multiscale collaboration and the man-agement of avatar position in direct avatar manipulationwhich illustrates cross-scale data coordination

431 Scene Graph of the Prototype Systemand Scaling Control The scene graph of the systemis shown in Figure 10 Each Java 3D scene graph has aVirtual Universe object as the root A Locale object at-taches scene graphs to the Virtual Universe objectBranchGroup (BG) nodes collect such objects as geom-etry behavior light sound and so forth A Transform-Group (TG) node defines the coordinates of objectsunder it

The system has two subscene graphs the object scenegraph which is the left child of the Locale object andthe view scene graph which is the right child The ob-ject scene graph collects all objects and behavior nodesThe view scene graph defines the viewing behaviors

To change a userrsquos interaction scale is to change thevalues of the userrsquos interaction parameters relative to thesize of the virtual world Thus scale change can be im-plemented in two ways (1) scaling the virtual world upand down and keeping all interaction parameters un-

touched or (2) changing all interaction parameters andkeeping the virtual world untouched Mathematicallythey are equivalent We adopted the first approach be-cause scaling the virtual world up and down can bedone by modifying just one parameter the scaling fac-tor Si of the top TG in the object scene graph This ismuch simpler than the second approach which involvesthe modification of complicated view parameters

432 Scale-Based Semantic RepresentationsIn the implementation of scale-based semantic represen-tations we adopted a design that uses a delegate objectto wrap all representations and that delivers appropriaterepresentations based on the value of the controllingvariable (Fox 1998) A scale-based semantic representa-tion node is extended from the Java 3D Switch objectEach node has a set of child representations and an or-dered array that defines the scale range for each repre-sentation (Figure 11)

433 Animating View Transition in Collabo-rative View Sharing The key issue in making a view-transition animation is to interpolate two views InmCVEs a view can be uniquely determined by its viewposition P its view orientation O and its view scale Sand be written as V(P O S)

Given two views V0 V(P0 O 0 S0) and V1 V(P1O 1 S1) to implement a view-transition animation is to

Figure 10 Scene graph of the prototype system

Zhang and Furnas 41

choose a path that links V0 and V1 and to set the trajec-tory of viewpoint movement view direction along thepath and observation scale along the path View-pathand view-orientation interpolation techniques are ma-ture (Parent 2001 Watt amp Policarpo 2001) so thisresearch focused only on the interpolation of view scale

A scale-interpolation function could be linear loga-rithmic or in other forms We chose a logarithmic func-tion because it gives users a constant relative rate ofchange in their views Interpolating the view scale loga-rithmically can give users a familiar experience

Assume the animation includes n frames from V0 toV1 The view scale of the ith frame (0 i n) in a log-arithmic interpolation function can be written as

Si Si S0 (1)

where

S S1

S01n

(2)

With corresponding P i and O i from interpolationfunctions for view position and view orientation theview of the ith frame Vi V(P i O i Si) can be ob-tained Figure 12 shows two views seen in Figure 7 asFrame 0 and Frame n and other interpolated viewframes (Due to space limitations we have shown onlyfour frames used for the animation) Showing theseframes successively with a reasonable frame rate can pro-duce an animation linking these two divergent views

434 Cross-Scale Data Consistency and Ava-tar Position in Direct Avatar Manipulation Al-though changing the scaling factor of the top TG in theobject scene graph simplifies the scaling operation such

an approach poses a challenge in cross-scale data consis-tency In CVEs all users usually need to see the sameobject appearing at the same spatial location In conven-tional CVEs this can be easily achieved because an ob-jectrsquos coordinates in different usersrsquo worlds are often thesame In mCVEs however when users are at differentscales their virtual worlds are rescaled according to thescaling factors of the top TG in their own object scenegraphs The same object will have different local coordi-nates in different usersrsquo worlds When the object ismoved by a user the objectrsquos local coordinates in thisuserrsquos world cannot be used to update the objectrsquos newposition in all other usersrsquo worlds

In the implementation we used normalized coordi-nates which are calculated as the ratio of the object lo-cal coordinates in the userrsquos world to the userrsquos scaleLet Pl(xl yl zl) and Sl denote the local coordinates andthe userrsquos scale respectively The normalized coordi-nates Pn can be written as

Pn PlSl (3)

By doing so the same object will always have the samePn This Pn is used to define the coordinates of the ob-ject in every userrsquos object scene graph Modifying thespatial location of an object in a userrsquos world leads to anew Pn All users can use this Pn to update the coordi-nates of the object in their object scene graphs There-fore Pn enforces data consistency across scale Becausethe scaling factor is on the top of the scene graph theobjectrsquos final location in a userrsquos world is Pl the productof Pn and this userrsquos scale Sl With different Sl users willbe able to see the world rendered at different scales

An avatarrsquos position can be managed similarly butwith a small difference An avatar object differs fromother ordinary objects in a way that in its host userrsquos

Figure 11 Scale-based semantic representation node

Figure 12 Frames of a dynamic view-transition animation

42 PRESENCE VOLUME 14 NUMBER 1

world the coordinates of the avatar are affiliated withthe view scene graph rather than the object scenegraph so that the avatar always moves with the view Inother usersrsquo worlds however the avatar object is underthe object scene graphs Figure 13(a) is the scene graphof the world of a user say A who interacts with the vir-tual environment at a location Pa and with a scale SaFigure 13(b) is the scene graph of the world of anotheruser say B User Arsquos avatar is in User Brsquos object graphBecause Pa is not governed by Sa the normalized coor-dinates of the avatar in User Brsquos object graph should bewritten as

PnPaSa (4)

The body size of User Arsquos avatar changes with thescale of User A so a scale parameter is also needed todefine the size of User Arsquos avatar in User Brsquos objectscene graph so that User B can be aware of User Arsquosscale This scale should also be normalized Let Sn de-note the normalized scale where Sn is simply the recip-rocal of Sa

Sn 1Sa (5)

This is because the avatar body changes inversely pro-portional to the scale factor Sa (shrinking the world isequivalent to growing the avatar body and growing theworld is equivalent to shrinking the avatar body)

The implementation of other cross-scale tools includ-ing direct avatar manipulation also uses Pn and Sn todefine the coordinates and size of an avatar entity Thisapproach guarantees the consistency of the avatar posi-tion and size in all usersrsquo views at different scales

5 Experimental Evaluation

We evaluated multiscale collaboration in subjecttests An experiment was designed to assess the benefitsusers can get from an mCVE in accomplishing tasks re-quiring cross-scale coordination

51 Experiment Design and Procedure

Subject tasks in the experiment involved searchingfor a ldquobombrdquo on a square ground plane (2000 2000m2) with a distinctly shaped building in each corner(square hexagon octagon and circle as seen fromabove) Each had a height of 12 m and a base of about6400 m2 On the ground behind each building was aunit cube (1 m3) containing a unique text name andsmiley face (about 05 05 m2) The bomb was insideone of these four cubes In the test subjects were placedin the middle of the square plane They had a defaulteye level of 168 m and a default moving step of 1 m inthe virtual environment The shape of the building thatthe bomb was nearby was known in advance and sub-jects needed to find that building locate the bomb boxand key in the name of the box to defuse the bomb

A 2 2 2 factorial design that has six treatmentstwo noncollaboration and four collaboration wasadopted (Table 1) For the two noncollaboration treat-ments VE is just a conventional 3D virtual environ-ment and M-VE is a VE enhanced by multiscale toolsspecifically it allows users to change their interactionscales eye level and speed Among four collaborationtreatments CVE a conventional collaborative virtualenvironment is the only one without multiscale toolsThe other three treatments all equipped with multiscale

Figure 13 Two scene graphs for the same avatar

Table 1 Six Treatments in a 2 2 2 Design

Noncollaboration Collaboration

Nonmultiscale VE CVEMultiscale M-VE NR

GUIDEMOVE

Zhang and Furnas 43

tools differ in the assignment of subject task roles (be-ing a giant or a normal person) and in the way subjectsaffect each otherrsquos work across scales In one treatmentthe roles of subjects are not predefined and subjects canchoose their own interaction scales as desired This envi-ronment is labeled as NR (No Role) The other twomCVEs both assign one subject to be a giant and theother a normal person so that subjects can change theirinteraction scales only within a limited range In onesuch environment the giant is permitted to move thepartner directly and this treatment is labeled as MOVEAnother condition allows the giant to guide the move-ment of the partner only verbally and is denoted asGUIDE

Recruited through email 24 students paired in 12groups participated in the experiment In noncollabora-tion treatments they worked on their own and in col-laboration treatments they communicated through anaudio channel The performances were measured bytask-completion time

52 Results

An ANOVA analysis of data from four treatments(VE M-VE CVE and MOVE) shows main effects ofboth collaboration (F170 1298 p 0001) andmultiscale (F170 7090 p 00001) Interaction isnot significant (F170 187 p 176) as seen in Fig-ure 14 Subjects performed best in MOVE where theycould take full advantage of multiscale and collabora-tion and they did worst in VE where there was no helpat all

Subjects used different strategies in the different con-ditions Without multiscale tools subjects had to goaround and count the number of sides of all four build-ings to find the target a very time-consuming processWith multiscale subjects could increase their sizes to seebuilding shapes from above and approach the bombquickly so the time required was be reduced signifi-cantly

The performance difference among the three mCVEsalso indicates the importance of different collaborationsupports In the NR treatment subjects had to assumetheir roles via an expensive negotiation process In the

GUIDE treatment the giant subject who could see thebuilding shape must coordinate with the partner aboutnavigation verbally through a costly grounding process(Clark amp Brennan 1991) In MOVE however the all-seeing giant could actually move the partner to the des-tination quickly and bypassed grounding processes

6 Discussion

In this paper we have introduced a novel interac-tive environment multiscale collaborative virtual envi-ronment which gives users the freedom to control theirinteraction scales in collaboration Multiscale collabora-tion can help better allocate scarce scale resources andcombine different expertise and action capabilities ofindividual users in their work with large structures Wehave also discussed the implications of multiscale tech-nology for user interaction and the design and develop-ment of a Java-based mCVE system Our subject testshave shown that multiscale collaboration can facilitatepeoplersquos work that requires cross-scale information andcollaboration

Research reported in this paper has largely focused ongeneral interaction issues in mCVEs Designs have beenlimited to generic tools to support multiscale collabora-tion and experimental study has focused on generaltasks We expect that multiscale collaboration tools canbe used to support peoplersquos specific work in the realworld To achieve this goal the design of multiscale

Figure 14 Time (in seconds) for the six treatments

44 PRESENCE VOLUME 14 NUMBER 1

tools should support domain-specific tasks One of ourongoing research projects focuses on studying how multi-scale collaboration can support research in materials sci-ence and engineering We are investigating the needs ofmaterials scientists for multiscale collaboration and willdesign and implement tools that support these needsThe effectiveness and usability of these tools will also beevaluated when they are ready

At the same time we are interested in behavior issuesin mCVEs Some important research questions concern-ing user behavior in mCVEs have not been studiedthoroughly It is still unclear when and how users canbenefit from collaborative object manipulation acrossscale whether the designed dynamic view-transitiontechnique can indeed improve cross-scale informationsharing what mechanism is needed to help users bettermanage cross-scale collaboration overheads and soforth In addition it has been found that multiscale alsobrings some new challenges in social interactions(Zhang amp Furnas 2002) and efforts are needed tostudy what can be done to address these interaction is-sues Research in this direction will deepen our under-standing of mCVEs and help to design better tools tosupport multiscale collaboration

Acknowledgments

This research was funded in part by Microsoft Research

References

Ahl V amp Allen T (1996) Hierarchy theory A vision vocab-ulary and epistemology New York Columbia UniversityPress

Alexander C Ishikawa S amp Silverstein M (1977) A pat-tern language Oxford UK Oxford University Press

Bederson B amp Hollan J (1994) Pad A zoominggraphical interface for exploring alternate interface physicsProceedings of ACM UIST rsquo94 17ndash26

Benford S Bowers J Fahlen L Greenhalgh C amp Snow-don D (1995) User embodiment in collaborative virtualenvironments Proceedings of ACM CHI rsquo95 242ndash249

Benford S Snowdon D Colebourne A OrsquoBrien J ampRodden T (1997) Informing the design of collaborativevirtual environments Proceedings of ACM GROUP rsquo97 71ndash80

Carlsson C amp Hagsand O (1993) DIVEmdashA platform formulti-user virtual environments Computer and Graphics17(6) 663ndash669

Clark H amp Brennan S (1991) Grounding in communica-tion In L B Resnick J Levine amp S D Behrend (Eds)Perspectives on socially shared cognition (pp 127ndash149)Washington DC American Psychological Association

Dix A J (1994) Computer-supported cooperative workmdashAframework In D Rosenburg amp C Hutchison (Eds) De-sign Issues in CSCW (pp 23ndash37) Berlin Springer-Verlag

EPFL Geneva IIS TNO Lancaster UCL et al (1997)Guidelines for building CVE applications COVEN ProjectRetrieved November 21 2002 from httpwwwcsuclacukstaffmslatervrCovenPublicDeliverablesdel26PDF

Fox D (1998) Tabula rasa A multi-scale user interface sys-tem Unpublished doctoral dissertation New York Univer-sity

Furnas G (1986) Generalized fisheye views Proceedings ofACM CHI rsquo86 16ndash23

Gaver W Sellen A Heath C amp Luff P (1993) One isnot enough Multiple views in a media space Proceedings ofACM INTERCHI rsquo93 335ndash341

Hollan J amp Stornetta S (1992) Beyond being there Pro-ceedings of ACM CHI rsquo92 119ndash125

Koike H amp Yoshihara H (1993) Fractal approaches forvisualizing huge hierarchies Proceedings of IEEE VL rsquo9355ndash60

Lamping J Rao R amp Pirolli P (1995) A focus contexttechnique based on hyperbolic geometry for visualizinglarge hierarchies Proceedings of ACM CHI rsquo95 401ndash408

Leigh J Johnson A amp DeFanti T (1996) CALVIN Animmersimedia design environment utilizing heterogeneousperspectives Proceedings of IEEE International Conferenceon Multimedia Computing and Systems 20ndash23

Lieberman H (1994) Powers of ten thousand Navigating inlarge information spaces Proceedings of ACM UIST rsquo9415ndash16

Mackinlay J Card S amp Robertson G (1990) Rapid con-trolled movement through a virtual 3D workspace Proceed-ings of ACM SIGGRAPH rsquo90 171ndash176

Mine M Brooks F amp Sequin C (1997) Moving objects

Zhang and Furnas 45

in space Exploiting proprioception in virtual-environmentinteraction Proceedings of ACM SIGGRAPH rsquo97 19ndash26

Morrison P amp Morrison P (1994) Powers of ten About therelative size of things in the universe and the effect of addinganother zero San Francisco W H Freeman

Parent R (2001) Computer animation Algorithms and tech-niques San Francisco Morgan-Kaufmann

Pattee H (1973) Hierarchy theory The challenge of complexsystems New York George Braziller

Perlin K amp Fox D (1993) Pad An alternative approach tothe computer interface Proceedings of ACM SIGGRAPHrsquo93 57ndash64

Pierce J Forsberg A Conway M Hong S Zeleznik Ramp Mine M (1997) Image plane interaction techniques in3D immersive environments Proceedings of the 1997 Sympo-sium on Interactive 3D Graphics 39ndash43

Poupyrev I Billinghurst M Weghorst S amp Ichikawa T(1996) The go-go interaction technique Non-linear map-ping for direct manipulation in VR Proceedings of ACMUIST rsquo96 79ndash80

Puppo E amp Scopigno R (1997) Simplification LOD multi-resolution Principles and applications Proceedings ofEUROGRAPHICS 16 (3)

Raab A amp Ruger M (1996) 3D-Zoom Interactive visual-ization of structures and relations in complex graphics InB Girod H Nieman amp H Seidel (Eds) 3D image analy-sis and synthesis (pp 125ndash132) Sankt Augustin GermanyInfix-Verlag

Robbins M (2001) Modeling and simulation of the failureand toughness of polymersubstrate bonds The second An-nual Multiscale ModelingmdashBridging the Gap Between Atom-istic and Meso Scales

Robertson G Card S amp Mackinlay J (1991) Informationvisualization using 3D interactive animation Proceedings ofACM CHI rsquo91 461ndash462

Robertson G amp Mackinlay J (1993) The document lensProceedings of ACM UIST rsquo93 101ndash110

Robertson G Mackinlay J amp Card S (1991) Cone treesAnimated 3D visualizations of hierarchical informationCommunications of the ACM 34 (2) p 89ndash194

Sarkar M amp Brown M H (1992) Graphical fisheye viewsof graphs Proceedings of ACM CHI rsquo92 83ndash91

Singhal S amp Zyda M (1999) Networked virtual environ-ments Design and implementation New York ACM Press

Snowdon D amp Jaa-Aro K (1997) A subjective virtual envi-ronment for collaborative information visualization VirtualReality Universe rsquo97 Retrieved May 23 2003 fromhttpwwwcyberedgecomvru_paperssnowdenhtm

Stappers P Gaver W amp Overbeeke C (2000) Beyond thelimits of real-time realism 2000 Chapter to appear in L JHettinger amp M W Haas (Eds) Psychological issues in thedesign and use of virtual and adaptive environment Mah-wah NJ Erlbaum

Stefik M Bobrow D Foster G Lanning S amp Tatar D(1987) WYSIWIS revised Early experiences with multiuserinterfaces ACM Transactions on Information Systems(TOIS) 5(2) 147ndash167

Stoakley R Conway M amp Pausch R (1995) Virtual real-ity on a WIM Interactive worlds in miniature Proceedingsof ACM CHI rsquo95 265ndash272

Sutherland I (1965) The ultimate display Proceedings ofIFIP Congress 2

Tromp J G (1999) Usability design for CVEs COVENProject Retrieved November 21 2002 from httpcovenlancsacuk4deliverablesdel29pdf

Waters R Anderson D Barrus J Brogan D Casey MMcKeown S et al (1997) Diamond park and SPLINESocial virtual reality with 3D animation spoken interactionand runtime extendibility Presence Teleoperators and Vir-tual Environments 6(4) 461ndash481

Watt A amp Policarpo F (2001) 3D games Animation andadvanced real-time rendering New York Addison-Wesley

Zhang X amp Furnas G (2002) Social interactions in multi-scale CVEs Proceedings of ACM CVE 2002 31ndash38

46 PRESENCE VOLUME 14 NUMBER 1

be problematic It would demand tremendous effort tochange the view of the large structure even slightlyOther interaction parameters such as reachable distanceand manipulation accuracy should be similarly coordi-nated

Thus mVEs provide users with new interaction capa-bilities that are usually unavailable in conventional vir-tual environments In mVEs users can interact withvirtual worlds at multiple scale levels At any given scalesome things will be easy to see and manipulate whileothers will be either too large or too small to displayChanging the scale moves new things into the easilyvisible and manipulative range Coordinated observationand action capabilities help users to better interact withmultiscale structures Multiscale extends the designspace for information visualization and user interaction

22 Multiscale Collaboration

Multiscale collaboration can further facilitate userinteraction with multiscale structures In particular itcan help people with work on large structures that needmore labor on rapidly changing structures that demandparallel working and on complex structures that requiredomain knowledge beyond what an individual mayhave

221 Large Multiscale Structures Managinga large multiscale structure could be difficult for a singleuser Take a city-planning example Adding new designelements (eg paths or landmarks) may force plannersto move back and forth frequently between differentscales and check the impact of new additions on designat different levels For a large city continuously chang-ing scale could be very demanding and costly if there isjust one planner Multiscale collaboration can offersome help by placing individual planners with differentscales and having them focus on issues at their own scalelevel Planners can then work independently on prob-lems within their own scale scopes and collaborativelydeal with cross-scale issues Collaboration divides bigcross-scale problems into smaller within-scale ones thatcan be more easily handled by individuals

222 Rapidly Changing Multiscale StructuresIt could also be a challenge for a user to manage a struc-ture that changes rapidly however small that structuremay be Consider a scenario in which a small computernetwork managed by a system administrator is beinginfected by a worm virus The system administrator maynot be able to keep pace with the speed at which thevirus spreads If more system administrators are in-volved they can have individuals to check subnets atdifferent levels simultaneously and then take appropri-ate actions to stop the virus from further damaging theentire network Collaboration by supporting parallelworking improves peoplersquos response to problems atdifferent levels

223 Cross-Domain Multiscale StructuresChallenges may also come from the diversified knowl-edge and expertise required in understanding and man-aging structures spanning across the boundaries of sci-entific domains Understanding cross-domain multiscalestructures could be daunting for individuals For exam-ple in studying the strength of a new material acrossdifferent scale levels no single person would possess allthe required in-depth knowledge of chemistry materialsscience and mechanical engineering Chemists materi-als scientists and mechanical engineers may need towork together on this issue While they can focus onproblems in their own expertise domains individuallyworking together helps them see how properties at dif-ferent scale levels may affect each other and even dis-cover important issues that might be ignored withoutcross-scale collaboration Collaboration helps solvecross-scale problems by combining peoplersquos knowledgedomains from different scales

23 Multiscale Collaborative VirtualEnvironments

An mCVE is an mVE that supports collaborationIn mCVEs users can choose their own scales in the ob-servation of an object and work together on the sameobject at different scales A team of city planners forexample can have a regular planner of the normal hu-man size at street level and a ldquogiantrdquo planner at the city

Zhang and Furnas 35

level While each planner focuses on planning issues athis or her own level they can help each other on cross-scale issues The giant planner can take the street-levelplanner quickly to a distant destination and the street-level planner can help the partner place buildings orlandmarks at their exact locations Figure 6 shows twoplanners at two scales working together Their cross-scale collaboration could help make a plan that meetsthe requirements at various levels

As seen in Figure 6 each user is represented as anavatar in mCVEs with a scaled size corresponding tothe scale level of the user This avatar metaphor offerstwo benefits First avatars provide rich awareness infor-mation (eg user view locations viewing directionsbody gestures) that are critical to social interactions invirtual environments (Benford Bowers Fahlen Green-halgh amp Snowdon 1995) Second by yoking differentobservation and action parameters under the same ava-tar body avatars can help users better understand theassociation of different observation and action capabili-ties in mCVEs The eye level of a userrsquos avatar tells howfar the user can see the avatarrsquos arm length indicateshow far the user can reach the leg length shows howfast the user can move Seeing the size of a userrsquos avatarother users can tell at what scale the user is working andwhat the user can do

mCVEs could be a promising tool to help people dealwith multiscale structures but this new design paradigmraises many research issues ranging from the conceptualunderstanding of the implications of multiscale for userperception and action to the technical design of

mCVEs to the experimental evaluation of the effective-ness of mCVEs

24 Literature Review

Some research effort has been made to addressdesign issues related to user interaction with large struc-tures Furnas (1986) proposed a theoretical frameworkGeneralized Fisheye Views to manage large hierarchicalstructures by allocating more space for objects of inter-est This technique has been used to visualize varioustypes of data (Sarkar amp Brown 1992 Robertson ampMackinlay 1993 Lamping Rao amp Pirolli 1995 Raabamp Ruger 1996) Koike and Yoshihara (1993) used thefractal concept to expand the tree structure based on thedegrees of interest Cone Trees were built to present filehierarchies (Robertson Mackinlay amp Card 1991)These methods focused only on better using screenspace to deal with the size of information structures butgave little consideration to visualizing structures withdifferent representations based on user interactionscales Multiscale technology (Perlin amp Fox 1993 Bed-erson amp Hollan 1994 Lieberman 1994) providedearly inspiration for this research This technique wasusually regarded as an alternative user interface in 2DThis research extends it into 3D worlds and argues thatmultiscale technology is a powerful design that by al-lowing people to control their observation and actiondomains at different scales can augment our limitedcognitive capabilities

Scaling has been seen in some designs to support useractions in virtual environments Scaling tools were usedto increase reaching distance (Poupyrev BillinghurstWeghorst amp Ichikawa 1996 Mine Brooks amp Sequin1997 Pierce et al 1997) facilitate spatial knowledgeacquisition (Stoakley Conway amp Pausch 1995) andimprove navigation performances (Mackinlay Card ampRobertson 1990) These scaling tools usually focusedon only one interaction parameter and ignored the im-plications of the change of one parameter for all otherrelated perception and action While isolating individualinteraction parameters may work well in some situa-tions uncoordinated perception and action as discussed

Figure 6 Two city planners working at two different scales

36 PRESENCE VOLUME 14 NUMBER 1

previously could cause some problems in the interac-tion with large multiscale structures

Research literature on virtual environments support-ing collaboration is massive but consideration of cross-scale collaboration is rarely seen The most relevantproject is the CALVIN system (Leigh Johnson amp De-Fanti 1996) in which two users can collaborate on adesign project from two different perspectives In thesetwo perspectives objects were rendered with same rep-resentations and the system did not provide users withmultiple levels of abstraction at different scales Alsothe system gave users limited choices of working scalesand could not satisfy their needs for active and interac-tive control over scale across a much broader range

3 Design of mCVEs

There are some design guidelines for 3D collabo-rative virtual environments (CVEs) (Benford SnowdonColebourne OrsquoBrien amp Rodden 1997 EPFL et al1997 Tromp 1999) However they were constructedfor the design of conventional CVEs and common col-laboration actions in CVEs Applying them in designingmCVEs could be difficult because scale issues have notbeen considered in these guidelines In this research weadopt a design approach that starts from understandingthe implications of multiscale technology for user inter-actions and then derives design considerations based onthis understanding In this section efforts are made firstto analyze how multiscale may affect user interactionThen some conceptual designs are proposed to supportuser perception and action in mCVEs

31 Implications of Multiscale for UserInteraction

Multiscale could affect user interactions at differ-ent levels and in different ways Here we first discusshow multiscale can support the understanding of multi-scale structures Then we describe how it may impedeinformation sharing among users at different scales andhow it may facilitate cross-scale action

311 Understanding Multiscale StructuresAs discussed previously interacting with large structuresrequires users to obtain both detailed content and suffi-cient context information This well-known ldquocontent

contextrdquo problem is indeed a multiscale problem be-cause detailed content information and global contextinformation are usually distributed at different scale lev-els To understand complicated structures that presentdifferent features at different scales we usually breakphenomena into components and focus our attentionson issues within certain levels rather than all levels atonce (Pattee 1973 Ahl amp Allen 1996) and thenchoose appropriate tools (eg microscopes telescopes)for observation and analysis at different scales

In virtual environments multiscale technology couldhelp address this ldquofocus contextrdquo problem by allowingusers to control what they want to see and what they cansee from multiscale structures and then to manipulate theamount of context and content information displayed onthe screen Under different interaction scales objects canbe rendered at different sizes tiny objects can be madelarger for more detailed observation and huge objects canbe made smaller for a better big picture The same struc-ture can have different appearances at different scales in-forming users with the rich features of the structure asseen in Figure 5 The understanding of a complicatedstructure could be improved

312 Cross-Scale Information Sharing Col-laboration can facilitate peoplersquos work by improvingcommunications andor cooperation on shared artifacts(Dix 1994) Multiscale collaboration focuses more onartifacts than communications In such artifact-centeredcollaboration users often need to refer to shared arti-facts (Dix) Conventional CVEs are usually ldquoWhat-You-See-Is-What-I-Seerdquo (WYSIWIS) or ldquorelaxed WYSIWISrdquo(Stefik Bobrow Foster Lanning amp Tatar 1987) soknowing what others are referring to is fairly easymCVEs are not like this however Users may have verydifferent views at different scales and see totally differ-ent objects and structures Figure 7 shows the view of acity planner who oversees the whole city (a) and theview of another planner who checks buildings at thestreet level (b) These two views which are from very

Zhang and Furnas 37

different perspectives and show different levels of ab-straction of the city may make it difficult for two plan-ners to understand how their work can be related Aplanner can refer to a particular object in her view butthe other may not see it at all Multiscale technologymay hurt mutual understanding of objects in collabora-tion because of scale-related views

313 Cross-Scale Action Multiscale can alsobring some benefits to collaboration One of them isthat users can leverage their different action capabilitiesand different scopes of influence of their actions Takethe example of the two city planners again When thestreet-level planner needs to go to a distant place thecity-level planner with a great action domain can movethe partner easily to the destination and reduce the tasktime Or when the city-level planner wants to move toan exact location the street-level planner can move thepartnerrsquos avatar accurately because of high manipulationaccuracy It is mentioned previously that a similar tech-nique can be used in object manipulation Thus multi-scale collaboration can improve navigation and manipu-lation activities by allowing users to take advantage ofeach otherrsquos different action capabilities

32 Design Considerations

Since multiscale technology could affect user in-teraction both positively and negatively the design ofmCVEs focuses on maximizing the advantages of multi-scale technology and avoiding the disadvantages Thisprinciple leads to the following design considerations

321 Support for Better Understanding ofMultiscale Structures Multiscale structures shouldbe modeled in such a way that scale-dependent charac-teristics can be observed at different scale levels In thereal world multiscale structures are ubiquitous as seenin Figures 1 and 2 To observe these multiscale struc-tures we need only appropriate tools In 3D virtualworlds however what users can see is predesignedThus to visualize multiscale structures objects have tobe modeled and rendered in a multiscale way

322 Support for Cross-Scale InformationSharing Users should be able to share artifacts ofcommon interest with others across scale Tools such asmultiple views of the world (Gaver Sellen Heath ampLuff 1993) cannot help too much Such techniquesrequire some common objects in different views as ref-erences but usersrsquo very subjective views in mCVEs maymake it difficult to find such references It is a challengeto build mutual understanding of working contexts withsubjective views (Snowdon amp Jaa-Aro 1997) Usersneed help in understanding how their subjective viewsand objects in them are related to each other

323 Support for Cross-Scale Action Usersshould be allowed to get involved in othersrsquo work acrossscale so that they can leverage their different interactioncapabilities in collaboration They should be able to di-rectly manipulate objects that others are working onand move the avatars of other users Of course this in-tervention should be based on mutual agreements toavoid any unwanted consequences In the city-planningexample the city-level planner should not change thingsthe street-level planner is working on without consentOtherwise the street-level planner may get confusedand not understand what has happened

33 Conceptual Design

Based on these considerations some multiscaletools were designed Here discussions focus on designsthat are fundamentally important to mCVEs includingscale-based semantic representations that present scale-related characteristics of multiscale structures dynamic

Figure 7 Two views of a city at two scale levels

38 PRESENCE VOLUME 14 NUMBER 1

view transition that facilitates cross-scale informationsharing and direct avatar manipulation that supportscross-scale action

331 Scale-Based Semantic Representationsfor Multiscale Structures This design renders amultiscale structure with different representations atdifferent scales Figure 5 presents the visual results of asubstance under this technique Initially the substanceappears as a double helix in (a) The user changes thescale to increase the structure size and to see more de-tails in (b) Continuing the scaling the user finds a dif-ferent structure in an atom in (c) The user is informedof semantic characteristics of the substance at differentscales

This scale-based representation technique has oneproblem The change between two very different repre-sentations may confuse users For example the suddenappearance of the structure in Figure 5(c) may distractusers from their primary tasks To address this issue weadopted a fading tool (Bederson amp Hollan 1994Lieberman 1994) to smooth the transition betweendifferent representations Figure 8 shows a transitionprocess from a spherelike molecule from Figure 8(a) toa set of atoms inside it in Figure 8(d) Instead of jump-ing abruptly between different representations the dis-appearance of the molecule and the appearance of atomsare gradual The transition between two representationsis more easily comprehensible Cognition required inunderstanding the change of visual image can be kept atthe level of perceptual processing so that users can freetheir cognitive processing capacities for more compli-cated tasks (Robertson Card amp Mackinlay 1991)

Scale-based semantic representations can be seen asone member of the family of level of detail (LOD) tech-

niques which visualize the same structure with differentgeometric objects according to a particular interactionparameter but with a very different purpose Whilemost LOD techniques such as distance LOD primarilyconcern computation efficiency (Puppo amp Scopigno1997) scale-based semantic representation gives moreconsideration to the userrsquos interaction needs CommonLOD techniques render individual objects with differenttextures or geometries but scale-based semantic repre-sentation visualizes structures with different semanticabstractions LOD techniques can be integrated intoscale-based semantic representations to improve bothuser interaction and machine performance For examplein a city-planning project a city can have different struc-tures at the level of city district and street to informplanners semantically and distance LOD can be appliedin rendering individual objects in each structure to im-prove 3D computation

332 Dynamic View to Support Cross-ScaleInformation Sharing Design here addresses issuesconcerning how to help users at different scales to buildcommon understandings of their divergent views Onedesign choice is to use a dynamic view to narrow downthe view difference We used animation to show thetransition between two views Animation is generatedby interpolating the view positions view orientationsand view scales of two views For example if two usersrsquoviews at two different scales look like Figures 7(a) and7(b) an animation can be generated by adding interme-diate image frames between Figure 7(a) and 7(b) Theanimation can tell how one view can be transformed toanother and how objects in two views are related toeach other

333 Direct Avatar Manipulation to SupportCross-Scale Action To help users get involved inothersrsquo work so that they can take advantage of theirdifferent action capabilities we adopted a design choicethat allows direct manipulation on other usersrsquo avatarsWith such a tool planners in the above city-planningexample can easily cooperate on tasks that require largeaction domain and high action accuracy Of course thisdirect interference should be regulated The user who

Figure 8 Successive fading views in semantic representations

Zhang and Furnas 39

owns the avatar is always aware of such action and hascontrol over whether this action is allowed

The implication of such a direct manipulation of ava-tar body for user interface design deserves more atten-tion Under this design an avatar can ldquoaffordrdquo manipu-lation activities This is quite different from thetraditional role of an avatar merely as user embodimentin conventional CVEs The reason that no direct manip-ulation of an avatar is allowed in conventional CVEsmight be due to the influence of real life where the au-tonomy of human bodies is well respected To betterfacilitate peoplersquos work in virtual environments somereal-life constraints could be relaxed or even abandonedVirtual environments should not be treated as the repli-cation of the real world just as new technologies shouldnot be interpreted by old metaphors Direct manipula-tion of a userrsquos avatar by others is an effort to breakdown the metaphor of reality

4 System Implementation

A desktop mCVE prototype system consisting ofover 20000 lines of Java code was implemented Dis-cussions in this section focus on the choice of toolkits insoftware implementation general system architectureand the implementation of some multiscale tools

41 Choice of Toolkits

The design of virtual environments that supportcollaboration needs to consider issues in three aspects3D graphics rendering network communication man-agement and user interactions (Singhal amp Zyda 1999)It is not our interest to address issues related to low-level graphics rendering or network communicationsOur goal here is to support user interactions with multi-scale structures and with other users at different scalesTo achieve this interaction-oriented design goal weimplemented the mCVE prototype with two toolkitsJava 3D and Java Shared Data Toolkit (JSDT)

This hybrid approach allows a design focus on high-level interaction tools because Java 3D can handlegraphics rendering and JSDT can handle network com-

munications issues With the scene-graph managementof Java 3D we can construct virtual environments andstructures at the level of geometric objects rather thanat the level of vertices and polygons Java 3D allows theextension of new rendering behaviors which is not wellsupported by other CVE toolkits such as DIVE (Carls-son amp Hagsand 1993) and SPLINE (Waters et al1997) This flexibility is critical to the implementationof such tools as scale-based semantic representations anddynamic view transition Furthermore the easy integra-tion of other Java APIs such as Java Swing into Java3D can simplify user interface design JSDT is ideal forthe research because it provides flexible data deliveryservices in support of synchronous collaboration on theInternet JSDT supports basic abstraction of networksessions multicast message communications and con-currency control Thus the design can focus on opti-mizing the management of distributed 3D scenes andcross-scale communications One drawback of this hy-brid approach is the lack of those CVE services seen inSPLINE (Waters et al) Collaboration services have tobe built from scratch

42 System Layer and Architecture

The system layer of the prototype is shown in Fig-ure 9 The implementation focused on the two layersinside the dashed rectangle On top of Java 3D andJSDT are multiscale graphics services and collaboration

Figure 9 System layer of the mCVE prototype

40 PRESENCE VOLUME 14 NUMBER 1

services and above them are multiscale collaborationtools

Multiscale graphics services handle object-renderingissues and update user views based on their view posi-tions and interaction scales Collaboration services de-liver messages among distributed users and manage theconsistency of shared virtual environments Multiscalecollaboration services deal with such collaboration issuesas cross-scale information sharing and cross-scale avatarmanipulation These services lay the foundation for multi-scale tools

43 Implementation of Multiscale Tools

A set of multiscale collaboration tools were imple-mented and integrated into the prototype system Thissection introduces such tools as scaling control whichdistinguishes mCVEs from other CVEs scale-based se-mantic representations which visualize information in amultiscale way cross-scale dynamic view transitionwhich addresses the cross-scale information problemuniquely seen in multiscale collaboration and the man-agement of avatar position in direct avatar manipulationwhich illustrates cross-scale data coordination

431 Scene Graph of the Prototype Systemand Scaling Control The scene graph of the systemis shown in Figure 10 Each Java 3D scene graph has aVirtual Universe object as the root A Locale object at-taches scene graphs to the Virtual Universe objectBranchGroup (BG) nodes collect such objects as geom-etry behavior light sound and so forth A Transform-Group (TG) node defines the coordinates of objectsunder it

The system has two subscene graphs the object scenegraph which is the left child of the Locale object andthe view scene graph which is the right child The ob-ject scene graph collects all objects and behavior nodesThe view scene graph defines the viewing behaviors

To change a userrsquos interaction scale is to change thevalues of the userrsquos interaction parameters relative to thesize of the virtual world Thus scale change can be im-plemented in two ways (1) scaling the virtual world upand down and keeping all interaction parameters un-

touched or (2) changing all interaction parameters andkeeping the virtual world untouched Mathematicallythey are equivalent We adopted the first approach be-cause scaling the virtual world up and down can bedone by modifying just one parameter the scaling fac-tor Si of the top TG in the object scene graph This ismuch simpler than the second approach which involvesthe modification of complicated view parameters

432 Scale-Based Semantic RepresentationsIn the implementation of scale-based semantic represen-tations we adopted a design that uses a delegate objectto wrap all representations and that delivers appropriaterepresentations based on the value of the controllingvariable (Fox 1998) A scale-based semantic representa-tion node is extended from the Java 3D Switch objectEach node has a set of child representations and an or-dered array that defines the scale range for each repre-sentation (Figure 11)

433 Animating View Transition in Collabo-rative View Sharing The key issue in making a view-transition animation is to interpolate two views InmCVEs a view can be uniquely determined by its viewposition P its view orientation O and its view scale Sand be written as V(P O S)

Given two views V0 V(P0 O 0 S0) and V1 V(P1O 1 S1) to implement a view-transition animation is to

Figure 10 Scene graph of the prototype system

Zhang and Furnas 41

choose a path that links V0 and V1 and to set the trajec-tory of viewpoint movement view direction along thepath and observation scale along the path View-pathand view-orientation interpolation techniques are ma-ture (Parent 2001 Watt amp Policarpo 2001) so thisresearch focused only on the interpolation of view scale

A scale-interpolation function could be linear loga-rithmic or in other forms We chose a logarithmic func-tion because it gives users a constant relative rate ofchange in their views Interpolating the view scale loga-rithmically can give users a familiar experience

Assume the animation includes n frames from V0 toV1 The view scale of the ith frame (0 i n) in a log-arithmic interpolation function can be written as

Si Si S0 (1)

where

S S1

S01n

(2)

With corresponding P i and O i from interpolationfunctions for view position and view orientation theview of the ith frame Vi V(P i O i Si) can be ob-tained Figure 12 shows two views seen in Figure 7 asFrame 0 and Frame n and other interpolated viewframes (Due to space limitations we have shown onlyfour frames used for the animation) Showing theseframes successively with a reasonable frame rate can pro-duce an animation linking these two divergent views

434 Cross-Scale Data Consistency and Ava-tar Position in Direct Avatar Manipulation Al-though changing the scaling factor of the top TG in theobject scene graph simplifies the scaling operation such

an approach poses a challenge in cross-scale data consis-tency In CVEs all users usually need to see the sameobject appearing at the same spatial location In conven-tional CVEs this can be easily achieved because an ob-jectrsquos coordinates in different usersrsquo worlds are often thesame In mCVEs however when users are at differentscales their virtual worlds are rescaled according to thescaling factors of the top TG in their own object scenegraphs The same object will have different local coordi-nates in different usersrsquo worlds When the object ismoved by a user the objectrsquos local coordinates in thisuserrsquos world cannot be used to update the objectrsquos newposition in all other usersrsquo worlds

In the implementation we used normalized coordi-nates which are calculated as the ratio of the object lo-cal coordinates in the userrsquos world to the userrsquos scaleLet Pl(xl yl zl) and Sl denote the local coordinates andthe userrsquos scale respectively The normalized coordi-nates Pn can be written as

Pn PlSl (3)

By doing so the same object will always have the samePn This Pn is used to define the coordinates of the ob-ject in every userrsquos object scene graph Modifying thespatial location of an object in a userrsquos world leads to anew Pn All users can use this Pn to update the coordi-nates of the object in their object scene graphs There-fore Pn enforces data consistency across scale Becausethe scaling factor is on the top of the scene graph theobjectrsquos final location in a userrsquos world is Pl the productof Pn and this userrsquos scale Sl With different Sl users willbe able to see the world rendered at different scales

An avatarrsquos position can be managed similarly butwith a small difference An avatar object differs fromother ordinary objects in a way that in its host userrsquos

Figure 11 Scale-based semantic representation node

Figure 12 Frames of a dynamic view-transition animation

42 PRESENCE VOLUME 14 NUMBER 1

world the coordinates of the avatar are affiliated withthe view scene graph rather than the object scenegraph so that the avatar always moves with the view Inother usersrsquo worlds however the avatar object is underthe object scene graphs Figure 13(a) is the scene graphof the world of a user say A who interacts with the vir-tual environment at a location Pa and with a scale SaFigure 13(b) is the scene graph of the world of anotheruser say B User Arsquos avatar is in User Brsquos object graphBecause Pa is not governed by Sa the normalized coor-dinates of the avatar in User Brsquos object graph should bewritten as

PnPaSa (4)

The body size of User Arsquos avatar changes with thescale of User A so a scale parameter is also needed todefine the size of User Arsquos avatar in User Brsquos objectscene graph so that User B can be aware of User Arsquosscale This scale should also be normalized Let Sn de-note the normalized scale where Sn is simply the recip-rocal of Sa

Sn 1Sa (5)

This is because the avatar body changes inversely pro-portional to the scale factor Sa (shrinking the world isequivalent to growing the avatar body and growing theworld is equivalent to shrinking the avatar body)

The implementation of other cross-scale tools includ-ing direct avatar manipulation also uses Pn and Sn todefine the coordinates and size of an avatar entity Thisapproach guarantees the consistency of the avatar posi-tion and size in all usersrsquo views at different scales

5 Experimental Evaluation

We evaluated multiscale collaboration in subjecttests An experiment was designed to assess the benefitsusers can get from an mCVE in accomplishing tasks re-quiring cross-scale coordination

51 Experiment Design and Procedure

Subject tasks in the experiment involved searchingfor a ldquobombrdquo on a square ground plane (2000 2000m2) with a distinctly shaped building in each corner(square hexagon octagon and circle as seen fromabove) Each had a height of 12 m and a base of about6400 m2 On the ground behind each building was aunit cube (1 m3) containing a unique text name andsmiley face (about 05 05 m2) The bomb was insideone of these four cubes In the test subjects were placedin the middle of the square plane They had a defaulteye level of 168 m and a default moving step of 1 m inthe virtual environment The shape of the building thatthe bomb was nearby was known in advance and sub-jects needed to find that building locate the bomb boxand key in the name of the box to defuse the bomb

A 2 2 2 factorial design that has six treatmentstwo noncollaboration and four collaboration wasadopted (Table 1) For the two noncollaboration treat-ments VE is just a conventional 3D virtual environ-ment and M-VE is a VE enhanced by multiscale toolsspecifically it allows users to change their interactionscales eye level and speed Among four collaborationtreatments CVE a conventional collaborative virtualenvironment is the only one without multiscale toolsThe other three treatments all equipped with multiscale

Figure 13 Two scene graphs for the same avatar

Table 1 Six Treatments in a 2 2 2 Design

Noncollaboration Collaboration

Nonmultiscale VE CVEMultiscale M-VE NR

GUIDEMOVE

Zhang and Furnas 43

tools differ in the assignment of subject task roles (be-ing a giant or a normal person) and in the way subjectsaffect each otherrsquos work across scales In one treatmentthe roles of subjects are not predefined and subjects canchoose their own interaction scales as desired This envi-ronment is labeled as NR (No Role) The other twomCVEs both assign one subject to be a giant and theother a normal person so that subjects can change theirinteraction scales only within a limited range In onesuch environment the giant is permitted to move thepartner directly and this treatment is labeled as MOVEAnother condition allows the giant to guide the move-ment of the partner only verbally and is denoted asGUIDE

Recruited through email 24 students paired in 12groups participated in the experiment In noncollabora-tion treatments they worked on their own and in col-laboration treatments they communicated through anaudio channel The performances were measured bytask-completion time

52 Results

An ANOVA analysis of data from four treatments(VE M-VE CVE and MOVE) shows main effects ofboth collaboration (F170 1298 p 0001) andmultiscale (F170 7090 p 00001) Interaction isnot significant (F170 187 p 176) as seen in Fig-ure 14 Subjects performed best in MOVE where theycould take full advantage of multiscale and collabora-tion and they did worst in VE where there was no helpat all

Subjects used different strategies in the different con-ditions Without multiscale tools subjects had to goaround and count the number of sides of all four build-ings to find the target a very time-consuming processWith multiscale subjects could increase their sizes to seebuilding shapes from above and approach the bombquickly so the time required was be reduced signifi-cantly

The performance difference among the three mCVEsalso indicates the importance of different collaborationsupports In the NR treatment subjects had to assumetheir roles via an expensive negotiation process In the

GUIDE treatment the giant subject who could see thebuilding shape must coordinate with the partner aboutnavigation verbally through a costly grounding process(Clark amp Brennan 1991) In MOVE however the all-seeing giant could actually move the partner to the des-tination quickly and bypassed grounding processes

6 Discussion

In this paper we have introduced a novel interac-tive environment multiscale collaborative virtual envi-ronment which gives users the freedom to control theirinteraction scales in collaboration Multiscale collabora-tion can help better allocate scarce scale resources andcombine different expertise and action capabilities ofindividual users in their work with large structures Wehave also discussed the implications of multiscale tech-nology for user interaction and the design and develop-ment of a Java-based mCVE system Our subject testshave shown that multiscale collaboration can facilitatepeoplersquos work that requires cross-scale information andcollaboration

Research reported in this paper has largely focused ongeneral interaction issues in mCVEs Designs have beenlimited to generic tools to support multiscale collabora-tion and experimental study has focused on generaltasks We expect that multiscale collaboration tools canbe used to support peoplersquos specific work in the realworld To achieve this goal the design of multiscale

Figure 14 Time (in seconds) for the six treatments

44 PRESENCE VOLUME 14 NUMBER 1

tools should support domain-specific tasks One of ourongoing research projects focuses on studying how multi-scale collaboration can support research in materials sci-ence and engineering We are investigating the needs ofmaterials scientists for multiscale collaboration and willdesign and implement tools that support these needsThe effectiveness and usability of these tools will also beevaluated when they are ready

At the same time we are interested in behavior issuesin mCVEs Some important research questions concern-ing user behavior in mCVEs have not been studiedthoroughly It is still unclear when and how users canbenefit from collaborative object manipulation acrossscale whether the designed dynamic view-transitiontechnique can indeed improve cross-scale informationsharing what mechanism is needed to help users bettermanage cross-scale collaboration overheads and soforth In addition it has been found that multiscale alsobrings some new challenges in social interactions(Zhang amp Furnas 2002) and efforts are needed tostudy what can be done to address these interaction is-sues Research in this direction will deepen our under-standing of mCVEs and help to design better tools tosupport multiscale collaboration

Acknowledgments

This research was funded in part by Microsoft Research

References

Ahl V amp Allen T (1996) Hierarchy theory A vision vocab-ulary and epistemology New York Columbia UniversityPress

Alexander C Ishikawa S amp Silverstein M (1977) A pat-tern language Oxford UK Oxford University Press

Bederson B amp Hollan J (1994) Pad A zoominggraphical interface for exploring alternate interface physicsProceedings of ACM UIST rsquo94 17ndash26

Benford S Bowers J Fahlen L Greenhalgh C amp Snow-don D (1995) User embodiment in collaborative virtualenvironments Proceedings of ACM CHI rsquo95 242ndash249

Benford S Snowdon D Colebourne A OrsquoBrien J ampRodden T (1997) Informing the design of collaborativevirtual environments Proceedings of ACM GROUP rsquo97 71ndash80

Carlsson C amp Hagsand O (1993) DIVEmdashA platform formulti-user virtual environments Computer and Graphics17(6) 663ndash669

Clark H amp Brennan S (1991) Grounding in communica-tion In L B Resnick J Levine amp S D Behrend (Eds)Perspectives on socially shared cognition (pp 127ndash149)Washington DC American Psychological Association

Dix A J (1994) Computer-supported cooperative workmdashAframework In D Rosenburg amp C Hutchison (Eds) De-sign Issues in CSCW (pp 23ndash37) Berlin Springer-Verlag

EPFL Geneva IIS TNO Lancaster UCL et al (1997)Guidelines for building CVE applications COVEN ProjectRetrieved November 21 2002 from httpwwwcsuclacukstaffmslatervrCovenPublicDeliverablesdel26PDF

Fox D (1998) Tabula rasa A multi-scale user interface sys-tem Unpublished doctoral dissertation New York Univer-sity

Furnas G (1986) Generalized fisheye views Proceedings ofACM CHI rsquo86 16ndash23

Gaver W Sellen A Heath C amp Luff P (1993) One isnot enough Multiple views in a media space Proceedings ofACM INTERCHI rsquo93 335ndash341

Hollan J amp Stornetta S (1992) Beyond being there Pro-ceedings of ACM CHI rsquo92 119ndash125

Koike H amp Yoshihara H (1993) Fractal approaches forvisualizing huge hierarchies Proceedings of IEEE VL rsquo9355ndash60

Lamping J Rao R amp Pirolli P (1995) A focus contexttechnique based on hyperbolic geometry for visualizinglarge hierarchies Proceedings of ACM CHI rsquo95 401ndash408

Leigh J Johnson A amp DeFanti T (1996) CALVIN Animmersimedia design environment utilizing heterogeneousperspectives Proceedings of IEEE International Conferenceon Multimedia Computing and Systems 20ndash23

Lieberman H (1994) Powers of ten thousand Navigating inlarge information spaces Proceedings of ACM UIST rsquo9415ndash16

Mackinlay J Card S amp Robertson G (1990) Rapid con-trolled movement through a virtual 3D workspace Proceed-ings of ACM SIGGRAPH rsquo90 171ndash176

Mine M Brooks F amp Sequin C (1997) Moving objects

Zhang and Furnas 45

in space Exploiting proprioception in virtual-environmentinteraction Proceedings of ACM SIGGRAPH rsquo97 19ndash26

Morrison P amp Morrison P (1994) Powers of ten About therelative size of things in the universe and the effect of addinganother zero San Francisco W H Freeman

Parent R (2001) Computer animation Algorithms and tech-niques San Francisco Morgan-Kaufmann

Pattee H (1973) Hierarchy theory The challenge of complexsystems New York George Braziller

Perlin K amp Fox D (1993) Pad An alternative approach tothe computer interface Proceedings of ACM SIGGRAPHrsquo93 57ndash64

Pierce J Forsberg A Conway M Hong S Zeleznik Ramp Mine M (1997) Image plane interaction techniques in3D immersive environments Proceedings of the 1997 Sympo-sium on Interactive 3D Graphics 39ndash43

Poupyrev I Billinghurst M Weghorst S amp Ichikawa T(1996) The go-go interaction technique Non-linear map-ping for direct manipulation in VR Proceedings of ACMUIST rsquo96 79ndash80

Puppo E amp Scopigno R (1997) Simplification LOD multi-resolution Principles and applications Proceedings ofEUROGRAPHICS 16 (3)

Raab A amp Ruger M (1996) 3D-Zoom Interactive visual-ization of structures and relations in complex graphics InB Girod H Nieman amp H Seidel (Eds) 3D image analy-sis and synthesis (pp 125ndash132) Sankt Augustin GermanyInfix-Verlag

Robbins M (2001) Modeling and simulation of the failureand toughness of polymersubstrate bonds The second An-nual Multiscale ModelingmdashBridging the Gap Between Atom-istic and Meso Scales

Robertson G Card S amp Mackinlay J (1991) Informationvisualization using 3D interactive animation Proceedings ofACM CHI rsquo91 461ndash462

Robertson G amp Mackinlay J (1993) The document lensProceedings of ACM UIST rsquo93 101ndash110

Robertson G Mackinlay J amp Card S (1991) Cone treesAnimated 3D visualizations of hierarchical informationCommunications of the ACM 34 (2) p 89ndash194

Sarkar M amp Brown M H (1992) Graphical fisheye viewsof graphs Proceedings of ACM CHI rsquo92 83ndash91

Singhal S amp Zyda M (1999) Networked virtual environ-ments Design and implementation New York ACM Press

Snowdon D amp Jaa-Aro K (1997) A subjective virtual envi-ronment for collaborative information visualization VirtualReality Universe rsquo97 Retrieved May 23 2003 fromhttpwwwcyberedgecomvru_paperssnowdenhtm

Stappers P Gaver W amp Overbeeke C (2000) Beyond thelimits of real-time realism 2000 Chapter to appear in L JHettinger amp M W Haas (Eds) Psychological issues in thedesign and use of virtual and adaptive environment Mah-wah NJ Erlbaum

Stefik M Bobrow D Foster G Lanning S amp Tatar D(1987) WYSIWIS revised Early experiences with multiuserinterfaces ACM Transactions on Information Systems(TOIS) 5(2) 147ndash167

Stoakley R Conway M amp Pausch R (1995) Virtual real-ity on a WIM Interactive worlds in miniature Proceedingsof ACM CHI rsquo95 265ndash272

Sutherland I (1965) The ultimate display Proceedings ofIFIP Congress 2

Tromp J G (1999) Usability design for CVEs COVENProject Retrieved November 21 2002 from httpcovenlancsacuk4deliverablesdel29pdf

Waters R Anderson D Barrus J Brogan D Casey MMcKeown S et al (1997) Diamond park and SPLINESocial virtual reality with 3D animation spoken interactionand runtime extendibility Presence Teleoperators and Vir-tual Environments 6(4) 461ndash481

Watt A amp Policarpo F (2001) 3D games Animation andadvanced real-time rendering New York Addison-Wesley

Zhang X amp Furnas G (2002) Social interactions in multi-scale CVEs Proceedings of ACM CVE 2002 31ndash38

46 PRESENCE VOLUME 14 NUMBER 1

level While each planner focuses on planning issues athis or her own level they can help each other on cross-scale issues The giant planner can take the street-levelplanner quickly to a distant destination and the street-level planner can help the partner place buildings orlandmarks at their exact locations Figure 6 shows twoplanners at two scales working together Their cross-scale collaboration could help make a plan that meetsthe requirements at various levels

As seen in Figure 6 each user is represented as anavatar in mCVEs with a scaled size corresponding tothe scale level of the user This avatar metaphor offerstwo benefits First avatars provide rich awareness infor-mation (eg user view locations viewing directionsbody gestures) that are critical to social interactions invirtual environments (Benford Bowers Fahlen Green-halgh amp Snowdon 1995) Second by yoking differentobservation and action parameters under the same ava-tar body avatars can help users better understand theassociation of different observation and action capabili-ties in mCVEs The eye level of a userrsquos avatar tells howfar the user can see the avatarrsquos arm length indicateshow far the user can reach the leg length shows howfast the user can move Seeing the size of a userrsquos avatarother users can tell at what scale the user is working andwhat the user can do

mCVEs could be a promising tool to help people dealwith multiscale structures but this new design paradigmraises many research issues ranging from the conceptualunderstanding of the implications of multiscale for userperception and action to the technical design of

mCVEs to the experimental evaluation of the effective-ness of mCVEs

24 Literature Review

Some research effort has been made to addressdesign issues related to user interaction with large struc-tures Furnas (1986) proposed a theoretical frameworkGeneralized Fisheye Views to manage large hierarchicalstructures by allocating more space for objects of inter-est This technique has been used to visualize varioustypes of data (Sarkar amp Brown 1992 Robertson ampMackinlay 1993 Lamping Rao amp Pirolli 1995 Raabamp Ruger 1996) Koike and Yoshihara (1993) used thefractal concept to expand the tree structure based on thedegrees of interest Cone Trees were built to present filehierarchies (Robertson Mackinlay amp Card 1991)These methods focused only on better using screenspace to deal with the size of information structures butgave little consideration to visualizing structures withdifferent representations based on user interactionscales Multiscale technology (Perlin amp Fox 1993 Bed-erson amp Hollan 1994 Lieberman 1994) providedearly inspiration for this research This technique wasusually regarded as an alternative user interface in 2DThis research extends it into 3D worlds and argues thatmultiscale technology is a powerful design that by al-lowing people to control their observation and actiondomains at different scales can augment our limitedcognitive capabilities

Scaling has been seen in some designs to support useractions in virtual environments Scaling tools were usedto increase reaching distance (Poupyrev BillinghurstWeghorst amp Ichikawa 1996 Mine Brooks amp Sequin1997 Pierce et al 1997) facilitate spatial knowledgeacquisition (Stoakley Conway amp Pausch 1995) andimprove navigation performances (Mackinlay Card ampRobertson 1990) These scaling tools usually focusedon only one interaction parameter and ignored the im-plications of the change of one parameter for all otherrelated perception and action While isolating individualinteraction parameters may work well in some situa-tions uncoordinated perception and action as discussed

Figure 6 Two city planners working at two different scales

36 PRESENCE VOLUME 14 NUMBER 1

previously could cause some problems in the interac-tion with large multiscale structures

Research literature on virtual environments support-ing collaboration is massive but consideration of cross-scale collaboration is rarely seen The most relevantproject is the CALVIN system (Leigh Johnson amp De-Fanti 1996) in which two users can collaborate on adesign project from two different perspectives In thesetwo perspectives objects were rendered with same rep-resentations and the system did not provide users withmultiple levels of abstraction at different scales Alsothe system gave users limited choices of working scalesand could not satisfy their needs for active and interac-tive control over scale across a much broader range

3 Design of mCVEs

There are some design guidelines for 3D collabo-rative virtual environments (CVEs) (Benford SnowdonColebourne OrsquoBrien amp Rodden 1997 EPFL et al1997 Tromp 1999) However they were constructedfor the design of conventional CVEs and common col-laboration actions in CVEs Applying them in designingmCVEs could be difficult because scale issues have notbeen considered in these guidelines In this research weadopt a design approach that starts from understandingthe implications of multiscale technology for user inter-actions and then derives design considerations based onthis understanding In this section efforts are made firstto analyze how multiscale may affect user interactionThen some conceptual designs are proposed to supportuser perception and action in mCVEs

31 Implications of Multiscale for UserInteraction

Multiscale could affect user interactions at differ-ent levels and in different ways Here we first discusshow multiscale can support the understanding of multi-scale structures Then we describe how it may impedeinformation sharing among users at different scales andhow it may facilitate cross-scale action

311 Understanding Multiscale StructuresAs discussed previously interacting with large structuresrequires users to obtain both detailed content and suffi-cient context information This well-known ldquocontent

contextrdquo problem is indeed a multiscale problem be-cause detailed content information and global contextinformation are usually distributed at different scale lev-els To understand complicated structures that presentdifferent features at different scales we usually breakphenomena into components and focus our attentionson issues within certain levels rather than all levels atonce (Pattee 1973 Ahl amp Allen 1996) and thenchoose appropriate tools (eg microscopes telescopes)for observation and analysis at different scales

In virtual environments multiscale technology couldhelp address this ldquofocus contextrdquo problem by allowingusers to control what they want to see and what they cansee from multiscale structures and then to manipulate theamount of context and content information displayed onthe screen Under different interaction scales objects canbe rendered at different sizes tiny objects can be madelarger for more detailed observation and huge objects canbe made smaller for a better big picture The same struc-ture can have different appearances at different scales in-forming users with the rich features of the structure asseen in Figure 5 The understanding of a complicatedstructure could be improved

312 Cross-Scale Information Sharing Col-laboration can facilitate peoplersquos work by improvingcommunications andor cooperation on shared artifacts(Dix 1994) Multiscale collaboration focuses more onartifacts than communications In such artifact-centeredcollaboration users often need to refer to shared arti-facts (Dix) Conventional CVEs are usually ldquoWhat-You-See-Is-What-I-Seerdquo (WYSIWIS) or ldquorelaxed WYSIWISrdquo(Stefik Bobrow Foster Lanning amp Tatar 1987) soknowing what others are referring to is fairly easymCVEs are not like this however Users may have verydifferent views at different scales and see totally differ-ent objects and structures Figure 7 shows the view of acity planner who oversees the whole city (a) and theview of another planner who checks buildings at thestreet level (b) These two views which are from very

Zhang and Furnas 37

different perspectives and show different levels of ab-straction of the city may make it difficult for two plan-ners to understand how their work can be related Aplanner can refer to a particular object in her view butthe other may not see it at all Multiscale technologymay hurt mutual understanding of objects in collabora-tion because of scale-related views

313 Cross-Scale Action Multiscale can alsobring some benefits to collaboration One of them isthat users can leverage their different action capabilitiesand different scopes of influence of their actions Takethe example of the two city planners again When thestreet-level planner needs to go to a distant place thecity-level planner with a great action domain can movethe partner easily to the destination and reduce the tasktime Or when the city-level planner wants to move toan exact location the street-level planner can move thepartnerrsquos avatar accurately because of high manipulationaccuracy It is mentioned previously that a similar tech-nique can be used in object manipulation Thus multi-scale collaboration can improve navigation and manipu-lation activities by allowing users to take advantage ofeach otherrsquos different action capabilities

32 Design Considerations

Since multiscale technology could affect user in-teraction both positively and negatively the design ofmCVEs focuses on maximizing the advantages of multi-scale technology and avoiding the disadvantages Thisprinciple leads to the following design considerations

321 Support for Better Understanding ofMultiscale Structures Multiscale structures shouldbe modeled in such a way that scale-dependent charac-teristics can be observed at different scale levels In thereal world multiscale structures are ubiquitous as seenin Figures 1 and 2 To observe these multiscale struc-tures we need only appropriate tools In 3D virtualworlds however what users can see is predesignedThus to visualize multiscale structures objects have tobe modeled and rendered in a multiscale way

322 Support for Cross-Scale InformationSharing Users should be able to share artifacts ofcommon interest with others across scale Tools such asmultiple views of the world (Gaver Sellen Heath ampLuff 1993) cannot help too much Such techniquesrequire some common objects in different views as ref-erences but usersrsquo very subjective views in mCVEs maymake it difficult to find such references It is a challengeto build mutual understanding of working contexts withsubjective views (Snowdon amp Jaa-Aro 1997) Usersneed help in understanding how their subjective viewsand objects in them are related to each other

323 Support for Cross-Scale Action Usersshould be allowed to get involved in othersrsquo work acrossscale so that they can leverage their different interactioncapabilities in collaboration They should be able to di-rectly manipulate objects that others are working onand move the avatars of other users Of course this in-tervention should be based on mutual agreements toavoid any unwanted consequences In the city-planningexample the city-level planner should not change thingsthe street-level planner is working on without consentOtherwise the street-level planner may get confusedand not understand what has happened

33 Conceptual Design

Based on these considerations some multiscaletools were designed Here discussions focus on designsthat are fundamentally important to mCVEs includingscale-based semantic representations that present scale-related characteristics of multiscale structures dynamic

Figure 7 Two views of a city at two scale levels

38 PRESENCE VOLUME 14 NUMBER 1

view transition that facilitates cross-scale informationsharing and direct avatar manipulation that supportscross-scale action

331 Scale-Based Semantic Representationsfor Multiscale Structures This design renders amultiscale structure with different representations atdifferent scales Figure 5 presents the visual results of asubstance under this technique Initially the substanceappears as a double helix in (a) The user changes thescale to increase the structure size and to see more de-tails in (b) Continuing the scaling the user finds a dif-ferent structure in an atom in (c) The user is informedof semantic characteristics of the substance at differentscales

This scale-based representation technique has oneproblem The change between two very different repre-sentations may confuse users For example the suddenappearance of the structure in Figure 5(c) may distractusers from their primary tasks To address this issue weadopted a fading tool (Bederson amp Hollan 1994Lieberman 1994) to smooth the transition betweendifferent representations Figure 8 shows a transitionprocess from a spherelike molecule from Figure 8(a) toa set of atoms inside it in Figure 8(d) Instead of jump-ing abruptly between different representations the dis-appearance of the molecule and the appearance of atomsare gradual The transition between two representationsis more easily comprehensible Cognition required inunderstanding the change of visual image can be kept atthe level of perceptual processing so that users can freetheir cognitive processing capacities for more compli-cated tasks (Robertson Card amp Mackinlay 1991)

Scale-based semantic representations can be seen asone member of the family of level of detail (LOD) tech-

niques which visualize the same structure with differentgeometric objects according to a particular interactionparameter but with a very different purpose Whilemost LOD techniques such as distance LOD primarilyconcern computation efficiency (Puppo amp Scopigno1997) scale-based semantic representation gives moreconsideration to the userrsquos interaction needs CommonLOD techniques render individual objects with differenttextures or geometries but scale-based semantic repre-sentation visualizes structures with different semanticabstractions LOD techniques can be integrated intoscale-based semantic representations to improve bothuser interaction and machine performance For examplein a city-planning project a city can have different struc-tures at the level of city district and street to informplanners semantically and distance LOD can be appliedin rendering individual objects in each structure to im-prove 3D computation

332 Dynamic View to Support Cross-ScaleInformation Sharing Design here addresses issuesconcerning how to help users at different scales to buildcommon understandings of their divergent views Onedesign choice is to use a dynamic view to narrow downthe view difference We used animation to show thetransition between two views Animation is generatedby interpolating the view positions view orientationsand view scales of two views For example if two usersrsquoviews at two different scales look like Figures 7(a) and7(b) an animation can be generated by adding interme-diate image frames between Figure 7(a) and 7(b) Theanimation can tell how one view can be transformed toanother and how objects in two views are related toeach other

333 Direct Avatar Manipulation to SupportCross-Scale Action To help users get involved inothersrsquo work so that they can take advantage of theirdifferent action capabilities we adopted a design choicethat allows direct manipulation on other usersrsquo avatarsWith such a tool planners in the above city-planningexample can easily cooperate on tasks that require largeaction domain and high action accuracy Of course thisdirect interference should be regulated The user who

Figure 8 Successive fading views in semantic representations

Zhang and Furnas 39

owns the avatar is always aware of such action and hascontrol over whether this action is allowed

The implication of such a direct manipulation of ava-tar body for user interface design deserves more atten-tion Under this design an avatar can ldquoaffordrdquo manipu-lation activities This is quite different from thetraditional role of an avatar merely as user embodimentin conventional CVEs The reason that no direct manip-ulation of an avatar is allowed in conventional CVEsmight be due to the influence of real life where the au-tonomy of human bodies is well respected To betterfacilitate peoplersquos work in virtual environments somereal-life constraints could be relaxed or even abandonedVirtual environments should not be treated as the repli-cation of the real world just as new technologies shouldnot be interpreted by old metaphors Direct manipula-tion of a userrsquos avatar by others is an effort to breakdown the metaphor of reality

4 System Implementation

A desktop mCVE prototype system consisting ofover 20000 lines of Java code was implemented Dis-cussions in this section focus on the choice of toolkits insoftware implementation general system architectureand the implementation of some multiscale tools

41 Choice of Toolkits

The design of virtual environments that supportcollaboration needs to consider issues in three aspects3D graphics rendering network communication man-agement and user interactions (Singhal amp Zyda 1999)It is not our interest to address issues related to low-level graphics rendering or network communicationsOur goal here is to support user interactions with multi-scale structures and with other users at different scalesTo achieve this interaction-oriented design goal weimplemented the mCVE prototype with two toolkitsJava 3D and Java Shared Data Toolkit (JSDT)

This hybrid approach allows a design focus on high-level interaction tools because Java 3D can handlegraphics rendering and JSDT can handle network com-

munications issues With the scene-graph managementof Java 3D we can construct virtual environments andstructures at the level of geometric objects rather thanat the level of vertices and polygons Java 3D allows theextension of new rendering behaviors which is not wellsupported by other CVE toolkits such as DIVE (Carls-son amp Hagsand 1993) and SPLINE (Waters et al1997) This flexibility is critical to the implementationof such tools as scale-based semantic representations anddynamic view transition Furthermore the easy integra-tion of other Java APIs such as Java Swing into Java3D can simplify user interface design JSDT is ideal forthe research because it provides flexible data deliveryservices in support of synchronous collaboration on theInternet JSDT supports basic abstraction of networksessions multicast message communications and con-currency control Thus the design can focus on opti-mizing the management of distributed 3D scenes andcross-scale communications One drawback of this hy-brid approach is the lack of those CVE services seen inSPLINE (Waters et al) Collaboration services have tobe built from scratch

42 System Layer and Architecture

The system layer of the prototype is shown in Fig-ure 9 The implementation focused on the two layersinside the dashed rectangle On top of Java 3D andJSDT are multiscale graphics services and collaboration

Figure 9 System layer of the mCVE prototype

40 PRESENCE VOLUME 14 NUMBER 1

services and above them are multiscale collaborationtools

Multiscale graphics services handle object-renderingissues and update user views based on their view posi-tions and interaction scales Collaboration services de-liver messages among distributed users and manage theconsistency of shared virtual environments Multiscalecollaboration services deal with such collaboration issuesas cross-scale information sharing and cross-scale avatarmanipulation These services lay the foundation for multi-scale tools

43 Implementation of Multiscale Tools

A set of multiscale collaboration tools were imple-mented and integrated into the prototype system Thissection introduces such tools as scaling control whichdistinguishes mCVEs from other CVEs scale-based se-mantic representations which visualize information in amultiscale way cross-scale dynamic view transitionwhich addresses the cross-scale information problemuniquely seen in multiscale collaboration and the man-agement of avatar position in direct avatar manipulationwhich illustrates cross-scale data coordination

431 Scene Graph of the Prototype Systemand Scaling Control The scene graph of the systemis shown in Figure 10 Each Java 3D scene graph has aVirtual Universe object as the root A Locale object at-taches scene graphs to the Virtual Universe objectBranchGroup (BG) nodes collect such objects as geom-etry behavior light sound and so forth A Transform-Group (TG) node defines the coordinates of objectsunder it

The system has two subscene graphs the object scenegraph which is the left child of the Locale object andthe view scene graph which is the right child The ob-ject scene graph collects all objects and behavior nodesThe view scene graph defines the viewing behaviors

To change a userrsquos interaction scale is to change thevalues of the userrsquos interaction parameters relative to thesize of the virtual world Thus scale change can be im-plemented in two ways (1) scaling the virtual world upand down and keeping all interaction parameters un-

touched or (2) changing all interaction parameters andkeeping the virtual world untouched Mathematicallythey are equivalent We adopted the first approach be-cause scaling the virtual world up and down can bedone by modifying just one parameter the scaling fac-tor Si of the top TG in the object scene graph This ismuch simpler than the second approach which involvesthe modification of complicated view parameters

432 Scale-Based Semantic RepresentationsIn the implementation of scale-based semantic represen-tations we adopted a design that uses a delegate objectto wrap all representations and that delivers appropriaterepresentations based on the value of the controllingvariable (Fox 1998) A scale-based semantic representa-tion node is extended from the Java 3D Switch objectEach node has a set of child representations and an or-dered array that defines the scale range for each repre-sentation (Figure 11)

433 Animating View Transition in Collabo-rative View Sharing The key issue in making a view-transition animation is to interpolate two views InmCVEs a view can be uniquely determined by its viewposition P its view orientation O and its view scale Sand be written as V(P O S)

Given two views V0 V(P0 O 0 S0) and V1 V(P1O 1 S1) to implement a view-transition animation is to

Figure 10 Scene graph of the prototype system

Zhang and Furnas 41

choose a path that links V0 and V1 and to set the trajec-tory of viewpoint movement view direction along thepath and observation scale along the path View-pathand view-orientation interpolation techniques are ma-ture (Parent 2001 Watt amp Policarpo 2001) so thisresearch focused only on the interpolation of view scale

A scale-interpolation function could be linear loga-rithmic or in other forms We chose a logarithmic func-tion because it gives users a constant relative rate ofchange in their views Interpolating the view scale loga-rithmically can give users a familiar experience

Assume the animation includes n frames from V0 toV1 The view scale of the ith frame (0 i n) in a log-arithmic interpolation function can be written as

Si Si S0 (1)

where

S S1

S01n

(2)

With corresponding P i and O i from interpolationfunctions for view position and view orientation theview of the ith frame Vi V(P i O i Si) can be ob-tained Figure 12 shows two views seen in Figure 7 asFrame 0 and Frame n and other interpolated viewframes (Due to space limitations we have shown onlyfour frames used for the animation) Showing theseframes successively with a reasonable frame rate can pro-duce an animation linking these two divergent views

434 Cross-Scale Data Consistency and Ava-tar Position in Direct Avatar Manipulation Al-though changing the scaling factor of the top TG in theobject scene graph simplifies the scaling operation such

an approach poses a challenge in cross-scale data consis-tency In CVEs all users usually need to see the sameobject appearing at the same spatial location In conven-tional CVEs this can be easily achieved because an ob-jectrsquos coordinates in different usersrsquo worlds are often thesame In mCVEs however when users are at differentscales their virtual worlds are rescaled according to thescaling factors of the top TG in their own object scenegraphs The same object will have different local coordi-nates in different usersrsquo worlds When the object ismoved by a user the objectrsquos local coordinates in thisuserrsquos world cannot be used to update the objectrsquos newposition in all other usersrsquo worlds

In the implementation we used normalized coordi-nates which are calculated as the ratio of the object lo-cal coordinates in the userrsquos world to the userrsquos scaleLet Pl(xl yl zl) and Sl denote the local coordinates andthe userrsquos scale respectively The normalized coordi-nates Pn can be written as

Pn PlSl (3)

By doing so the same object will always have the samePn This Pn is used to define the coordinates of the ob-ject in every userrsquos object scene graph Modifying thespatial location of an object in a userrsquos world leads to anew Pn All users can use this Pn to update the coordi-nates of the object in their object scene graphs There-fore Pn enforces data consistency across scale Becausethe scaling factor is on the top of the scene graph theobjectrsquos final location in a userrsquos world is Pl the productof Pn and this userrsquos scale Sl With different Sl users willbe able to see the world rendered at different scales

An avatarrsquos position can be managed similarly butwith a small difference An avatar object differs fromother ordinary objects in a way that in its host userrsquos

Figure 11 Scale-based semantic representation node

Figure 12 Frames of a dynamic view-transition animation

42 PRESENCE VOLUME 14 NUMBER 1

world the coordinates of the avatar are affiliated withthe view scene graph rather than the object scenegraph so that the avatar always moves with the view Inother usersrsquo worlds however the avatar object is underthe object scene graphs Figure 13(a) is the scene graphof the world of a user say A who interacts with the vir-tual environment at a location Pa and with a scale SaFigure 13(b) is the scene graph of the world of anotheruser say B User Arsquos avatar is in User Brsquos object graphBecause Pa is not governed by Sa the normalized coor-dinates of the avatar in User Brsquos object graph should bewritten as

PnPaSa (4)

The body size of User Arsquos avatar changes with thescale of User A so a scale parameter is also needed todefine the size of User Arsquos avatar in User Brsquos objectscene graph so that User B can be aware of User Arsquosscale This scale should also be normalized Let Sn de-note the normalized scale where Sn is simply the recip-rocal of Sa

Sn 1Sa (5)

This is because the avatar body changes inversely pro-portional to the scale factor Sa (shrinking the world isequivalent to growing the avatar body and growing theworld is equivalent to shrinking the avatar body)

The implementation of other cross-scale tools includ-ing direct avatar manipulation also uses Pn and Sn todefine the coordinates and size of an avatar entity Thisapproach guarantees the consistency of the avatar posi-tion and size in all usersrsquo views at different scales

5 Experimental Evaluation

We evaluated multiscale collaboration in subjecttests An experiment was designed to assess the benefitsusers can get from an mCVE in accomplishing tasks re-quiring cross-scale coordination

51 Experiment Design and Procedure

Subject tasks in the experiment involved searchingfor a ldquobombrdquo on a square ground plane (2000 2000m2) with a distinctly shaped building in each corner(square hexagon octagon and circle as seen fromabove) Each had a height of 12 m and a base of about6400 m2 On the ground behind each building was aunit cube (1 m3) containing a unique text name andsmiley face (about 05 05 m2) The bomb was insideone of these four cubes In the test subjects were placedin the middle of the square plane They had a defaulteye level of 168 m and a default moving step of 1 m inthe virtual environment The shape of the building thatthe bomb was nearby was known in advance and sub-jects needed to find that building locate the bomb boxand key in the name of the box to defuse the bomb

A 2 2 2 factorial design that has six treatmentstwo noncollaboration and four collaboration wasadopted (Table 1) For the two noncollaboration treat-ments VE is just a conventional 3D virtual environ-ment and M-VE is a VE enhanced by multiscale toolsspecifically it allows users to change their interactionscales eye level and speed Among four collaborationtreatments CVE a conventional collaborative virtualenvironment is the only one without multiscale toolsThe other three treatments all equipped with multiscale

Figure 13 Two scene graphs for the same avatar

Table 1 Six Treatments in a 2 2 2 Design

Noncollaboration Collaboration

Nonmultiscale VE CVEMultiscale M-VE NR

GUIDEMOVE

Zhang and Furnas 43

tools differ in the assignment of subject task roles (be-ing a giant or a normal person) and in the way subjectsaffect each otherrsquos work across scales In one treatmentthe roles of subjects are not predefined and subjects canchoose their own interaction scales as desired This envi-ronment is labeled as NR (No Role) The other twomCVEs both assign one subject to be a giant and theother a normal person so that subjects can change theirinteraction scales only within a limited range In onesuch environment the giant is permitted to move thepartner directly and this treatment is labeled as MOVEAnother condition allows the giant to guide the move-ment of the partner only verbally and is denoted asGUIDE

Recruited through email 24 students paired in 12groups participated in the experiment In noncollabora-tion treatments they worked on their own and in col-laboration treatments they communicated through anaudio channel The performances were measured bytask-completion time

52 Results

An ANOVA analysis of data from four treatments(VE M-VE CVE and MOVE) shows main effects ofboth collaboration (F170 1298 p 0001) andmultiscale (F170 7090 p 00001) Interaction isnot significant (F170 187 p 176) as seen in Fig-ure 14 Subjects performed best in MOVE where theycould take full advantage of multiscale and collabora-tion and they did worst in VE where there was no helpat all

Subjects used different strategies in the different con-ditions Without multiscale tools subjects had to goaround and count the number of sides of all four build-ings to find the target a very time-consuming processWith multiscale subjects could increase their sizes to seebuilding shapes from above and approach the bombquickly so the time required was be reduced signifi-cantly

The performance difference among the three mCVEsalso indicates the importance of different collaborationsupports In the NR treatment subjects had to assumetheir roles via an expensive negotiation process In the

GUIDE treatment the giant subject who could see thebuilding shape must coordinate with the partner aboutnavigation verbally through a costly grounding process(Clark amp Brennan 1991) In MOVE however the all-seeing giant could actually move the partner to the des-tination quickly and bypassed grounding processes

6 Discussion

In this paper we have introduced a novel interac-tive environment multiscale collaborative virtual envi-ronment which gives users the freedom to control theirinteraction scales in collaboration Multiscale collabora-tion can help better allocate scarce scale resources andcombine different expertise and action capabilities ofindividual users in their work with large structures Wehave also discussed the implications of multiscale tech-nology for user interaction and the design and develop-ment of a Java-based mCVE system Our subject testshave shown that multiscale collaboration can facilitatepeoplersquos work that requires cross-scale information andcollaboration

Research reported in this paper has largely focused ongeneral interaction issues in mCVEs Designs have beenlimited to generic tools to support multiscale collabora-tion and experimental study has focused on generaltasks We expect that multiscale collaboration tools canbe used to support peoplersquos specific work in the realworld To achieve this goal the design of multiscale

Figure 14 Time (in seconds) for the six treatments

44 PRESENCE VOLUME 14 NUMBER 1

tools should support domain-specific tasks One of ourongoing research projects focuses on studying how multi-scale collaboration can support research in materials sci-ence and engineering We are investigating the needs ofmaterials scientists for multiscale collaboration and willdesign and implement tools that support these needsThe effectiveness and usability of these tools will also beevaluated when they are ready

At the same time we are interested in behavior issuesin mCVEs Some important research questions concern-ing user behavior in mCVEs have not been studiedthoroughly It is still unclear when and how users canbenefit from collaborative object manipulation acrossscale whether the designed dynamic view-transitiontechnique can indeed improve cross-scale informationsharing what mechanism is needed to help users bettermanage cross-scale collaboration overheads and soforth In addition it has been found that multiscale alsobrings some new challenges in social interactions(Zhang amp Furnas 2002) and efforts are needed tostudy what can be done to address these interaction is-sues Research in this direction will deepen our under-standing of mCVEs and help to design better tools tosupport multiscale collaboration

Acknowledgments

This research was funded in part by Microsoft Research

References

Ahl V amp Allen T (1996) Hierarchy theory A vision vocab-ulary and epistemology New York Columbia UniversityPress

Alexander C Ishikawa S amp Silverstein M (1977) A pat-tern language Oxford UK Oxford University Press

Bederson B amp Hollan J (1994) Pad A zoominggraphical interface for exploring alternate interface physicsProceedings of ACM UIST rsquo94 17ndash26

Benford S Bowers J Fahlen L Greenhalgh C amp Snow-don D (1995) User embodiment in collaborative virtualenvironments Proceedings of ACM CHI rsquo95 242ndash249

Benford S Snowdon D Colebourne A OrsquoBrien J ampRodden T (1997) Informing the design of collaborativevirtual environments Proceedings of ACM GROUP rsquo97 71ndash80

Carlsson C amp Hagsand O (1993) DIVEmdashA platform formulti-user virtual environments Computer and Graphics17(6) 663ndash669

Clark H amp Brennan S (1991) Grounding in communica-tion In L B Resnick J Levine amp S D Behrend (Eds)Perspectives on socially shared cognition (pp 127ndash149)Washington DC American Psychological Association

Dix A J (1994) Computer-supported cooperative workmdashAframework In D Rosenburg amp C Hutchison (Eds) De-sign Issues in CSCW (pp 23ndash37) Berlin Springer-Verlag

EPFL Geneva IIS TNO Lancaster UCL et al (1997)Guidelines for building CVE applications COVEN ProjectRetrieved November 21 2002 from httpwwwcsuclacukstaffmslatervrCovenPublicDeliverablesdel26PDF

Fox D (1998) Tabula rasa A multi-scale user interface sys-tem Unpublished doctoral dissertation New York Univer-sity

Furnas G (1986) Generalized fisheye views Proceedings ofACM CHI rsquo86 16ndash23

Gaver W Sellen A Heath C amp Luff P (1993) One isnot enough Multiple views in a media space Proceedings ofACM INTERCHI rsquo93 335ndash341

Hollan J amp Stornetta S (1992) Beyond being there Pro-ceedings of ACM CHI rsquo92 119ndash125

Koike H amp Yoshihara H (1993) Fractal approaches forvisualizing huge hierarchies Proceedings of IEEE VL rsquo9355ndash60

Lamping J Rao R amp Pirolli P (1995) A focus contexttechnique based on hyperbolic geometry for visualizinglarge hierarchies Proceedings of ACM CHI rsquo95 401ndash408

Leigh J Johnson A amp DeFanti T (1996) CALVIN Animmersimedia design environment utilizing heterogeneousperspectives Proceedings of IEEE International Conferenceon Multimedia Computing and Systems 20ndash23

Lieberman H (1994) Powers of ten thousand Navigating inlarge information spaces Proceedings of ACM UIST rsquo9415ndash16

Mackinlay J Card S amp Robertson G (1990) Rapid con-trolled movement through a virtual 3D workspace Proceed-ings of ACM SIGGRAPH rsquo90 171ndash176

Mine M Brooks F amp Sequin C (1997) Moving objects

Zhang and Furnas 45

in space Exploiting proprioception in virtual-environmentinteraction Proceedings of ACM SIGGRAPH rsquo97 19ndash26

Morrison P amp Morrison P (1994) Powers of ten About therelative size of things in the universe and the effect of addinganother zero San Francisco W H Freeman

Parent R (2001) Computer animation Algorithms and tech-niques San Francisco Morgan-Kaufmann

Pattee H (1973) Hierarchy theory The challenge of complexsystems New York George Braziller

Perlin K amp Fox D (1993) Pad An alternative approach tothe computer interface Proceedings of ACM SIGGRAPHrsquo93 57ndash64

Pierce J Forsberg A Conway M Hong S Zeleznik Ramp Mine M (1997) Image plane interaction techniques in3D immersive environments Proceedings of the 1997 Sympo-sium on Interactive 3D Graphics 39ndash43

Poupyrev I Billinghurst M Weghorst S amp Ichikawa T(1996) The go-go interaction technique Non-linear map-ping for direct manipulation in VR Proceedings of ACMUIST rsquo96 79ndash80

Puppo E amp Scopigno R (1997) Simplification LOD multi-resolution Principles and applications Proceedings ofEUROGRAPHICS 16 (3)

Raab A amp Ruger M (1996) 3D-Zoom Interactive visual-ization of structures and relations in complex graphics InB Girod H Nieman amp H Seidel (Eds) 3D image analy-sis and synthesis (pp 125ndash132) Sankt Augustin GermanyInfix-Verlag

Robbins M (2001) Modeling and simulation of the failureand toughness of polymersubstrate bonds The second An-nual Multiscale ModelingmdashBridging the Gap Between Atom-istic and Meso Scales

Robertson G Card S amp Mackinlay J (1991) Informationvisualization using 3D interactive animation Proceedings ofACM CHI rsquo91 461ndash462

Robertson G amp Mackinlay J (1993) The document lensProceedings of ACM UIST rsquo93 101ndash110

Robertson G Mackinlay J amp Card S (1991) Cone treesAnimated 3D visualizations of hierarchical informationCommunications of the ACM 34 (2) p 89ndash194

Sarkar M amp Brown M H (1992) Graphical fisheye viewsof graphs Proceedings of ACM CHI rsquo92 83ndash91

Singhal S amp Zyda M (1999) Networked virtual environ-ments Design and implementation New York ACM Press

Snowdon D amp Jaa-Aro K (1997) A subjective virtual envi-ronment for collaborative information visualization VirtualReality Universe rsquo97 Retrieved May 23 2003 fromhttpwwwcyberedgecomvru_paperssnowdenhtm

Stappers P Gaver W amp Overbeeke C (2000) Beyond thelimits of real-time realism 2000 Chapter to appear in L JHettinger amp M W Haas (Eds) Psychological issues in thedesign and use of virtual and adaptive environment Mah-wah NJ Erlbaum

Stefik M Bobrow D Foster G Lanning S amp Tatar D(1987) WYSIWIS revised Early experiences with multiuserinterfaces ACM Transactions on Information Systems(TOIS) 5(2) 147ndash167

Stoakley R Conway M amp Pausch R (1995) Virtual real-ity on a WIM Interactive worlds in miniature Proceedingsof ACM CHI rsquo95 265ndash272

Sutherland I (1965) The ultimate display Proceedings ofIFIP Congress 2

Tromp J G (1999) Usability design for CVEs COVENProject Retrieved November 21 2002 from httpcovenlancsacuk4deliverablesdel29pdf

Waters R Anderson D Barrus J Brogan D Casey MMcKeown S et al (1997) Diamond park and SPLINESocial virtual reality with 3D animation spoken interactionand runtime extendibility Presence Teleoperators and Vir-tual Environments 6(4) 461ndash481

Watt A amp Policarpo F (2001) 3D games Animation andadvanced real-time rendering New York Addison-Wesley

Zhang X amp Furnas G (2002) Social interactions in multi-scale CVEs Proceedings of ACM CVE 2002 31ndash38

46 PRESENCE VOLUME 14 NUMBER 1

previously could cause some problems in the interac-tion with large multiscale structures

Research literature on virtual environments support-ing collaboration is massive but consideration of cross-scale collaboration is rarely seen The most relevantproject is the CALVIN system (Leigh Johnson amp De-Fanti 1996) in which two users can collaborate on adesign project from two different perspectives In thesetwo perspectives objects were rendered with same rep-resentations and the system did not provide users withmultiple levels of abstraction at different scales Alsothe system gave users limited choices of working scalesand could not satisfy their needs for active and interac-tive control over scale across a much broader range

3 Design of mCVEs

There are some design guidelines for 3D collabo-rative virtual environments (CVEs) (Benford SnowdonColebourne OrsquoBrien amp Rodden 1997 EPFL et al1997 Tromp 1999) However they were constructedfor the design of conventional CVEs and common col-laboration actions in CVEs Applying them in designingmCVEs could be difficult because scale issues have notbeen considered in these guidelines In this research weadopt a design approach that starts from understandingthe implications of multiscale technology for user inter-actions and then derives design considerations based onthis understanding In this section efforts are made firstto analyze how multiscale may affect user interactionThen some conceptual designs are proposed to supportuser perception and action in mCVEs

31 Implications of Multiscale for UserInteraction

Multiscale could affect user interactions at differ-ent levels and in different ways Here we first discusshow multiscale can support the understanding of multi-scale structures Then we describe how it may impedeinformation sharing among users at different scales andhow it may facilitate cross-scale action

311 Understanding Multiscale StructuresAs discussed previously interacting with large structuresrequires users to obtain both detailed content and suffi-cient context information This well-known ldquocontent

contextrdquo problem is indeed a multiscale problem be-cause detailed content information and global contextinformation are usually distributed at different scale lev-els To understand complicated structures that presentdifferent features at different scales we usually breakphenomena into components and focus our attentionson issues within certain levels rather than all levels atonce (Pattee 1973 Ahl amp Allen 1996) and thenchoose appropriate tools (eg microscopes telescopes)for observation and analysis at different scales

In virtual environments multiscale technology couldhelp address this ldquofocus contextrdquo problem by allowingusers to control what they want to see and what they cansee from multiscale structures and then to manipulate theamount of context and content information displayed onthe screen Under different interaction scales objects canbe rendered at different sizes tiny objects can be madelarger for more detailed observation and huge objects canbe made smaller for a better big picture The same struc-ture can have different appearances at different scales in-forming users with the rich features of the structure asseen in Figure 5 The understanding of a complicatedstructure could be improved

312 Cross-Scale Information Sharing Col-laboration can facilitate peoplersquos work by improvingcommunications andor cooperation on shared artifacts(Dix 1994) Multiscale collaboration focuses more onartifacts than communications In such artifact-centeredcollaboration users often need to refer to shared arti-facts (Dix) Conventional CVEs are usually ldquoWhat-You-See-Is-What-I-Seerdquo (WYSIWIS) or ldquorelaxed WYSIWISrdquo(Stefik Bobrow Foster Lanning amp Tatar 1987) soknowing what others are referring to is fairly easymCVEs are not like this however Users may have verydifferent views at different scales and see totally differ-ent objects and structures Figure 7 shows the view of acity planner who oversees the whole city (a) and theview of another planner who checks buildings at thestreet level (b) These two views which are from very

Zhang and Furnas 37

different perspectives and show different levels of ab-straction of the city may make it difficult for two plan-ners to understand how their work can be related Aplanner can refer to a particular object in her view butthe other may not see it at all Multiscale technologymay hurt mutual understanding of objects in collabora-tion because of scale-related views

313 Cross-Scale Action Multiscale can alsobring some benefits to collaboration One of them isthat users can leverage their different action capabilitiesand different scopes of influence of their actions Takethe example of the two city planners again When thestreet-level planner needs to go to a distant place thecity-level planner with a great action domain can movethe partner easily to the destination and reduce the tasktime Or when the city-level planner wants to move toan exact location the street-level planner can move thepartnerrsquos avatar accurately because of high manipulationaccuracy It is mentioned previously that a similar tech-nique can be used in object manipulation Thus multi-scale collaboration can improve navigation and manipu-lation activities by allowing users to take advantage ofeach otherrsquos different action capabilities

32 Design Considerations

Since multiscale technology could affect user in-teraction both positively and negatively the design ofmCVEs focuses on maximizing the advantages of multi-scale technology and avoiding the disadvantages Thisprinciple leads to the following design considerations

321 Support for Better Understanding ofMultiscale Structures Multiscale structures shouldbe modeled in such a way that scale-dependent charac-teristics can be observed at different scale levels In thereal world multiscale structures are ubiquitous as seenin Figures 1 and 2 To observe these multiscale struc-tures we need only appropriate tools In 3D virtualworlds however what users can see is predesignedThus to visualize multiscale structures objects have tobe modeled and rendered in a multiscale way

322 Support for Cross-Scale InformationSharing Users should be able to share artifacts ofcommon interest with others across scale Tools such asmultiple views of the world (Gaver Sellen Heath ampLuff 1993) cannot help too much Such techniquesrequire some common objects in different views as ref-erences but usersrsquo very subjective views in mCVEs maymake it difficult to find such references It is a challengeto build mutual understanding of working contexts withsubjective views (Snowdon amp Jaa-Aro 1997) Usersneed help in understanding how their subjective viewsand objects in them are related to each other

323 Support for Cross-Scale Action Usersshould be allowed to get involved in othersrsquo work acrossscale so that they can leverage their different interactioncapabilities in collaboration They should be able to di-rectly manipulate objects that others are working onand move the avatars of other users Of course this in-tervention should be based on mutual agreements toavoid any unwanted consequences In the city-planningexample the city-level planner should not change thingsthe street-level planner is working on without consentOtherwise the street-level planner may get confusedand not understand what has happened

33 Conceptual Design

Based on these considerations some multiscaletools were designed Here discussions focus on designsthat are fundamentally important to mCVEs includingscale-based semantic representations that present scale-related characteristics of multiscale structures dynamic

Figure 7 Two views of a city at two scale levels

38 PRESENCE VOLUME 14 NUMBER 1

view transition that facilitates cross-scale informationsharing and direct avatar manipulation that supportscross-scale action

331 Scale-Based Semantic Representationsfor Multiscale Structures This design renders amultiscale structure with different representations atdifferent scales Figure 5 presents the visual results of asubstance under this technique Initially the substanceappears as a double helix in (a) The user changes thescale to increase the structure size and to see more de-tails in (b) Continuing the scaling the user finds a dif-ferent structure in an atom in (c) The user is informedof semantic characteristics of the substance at differentscales

This scale-based representation technique has oneproblem The change between two very different repre-sentations may confuse users For example the suddenappearance of the structure in Figure 5(c) may distractusers from their primary tasks To address this issue weadopted a fading tool (Bederson amp Hollan 1994Lieberman 1994) to smooth the transition betweendifferent representations Figure 8 shows a transitionprocess from a spherelike molecule from Figure 8(a) toa set of atoms inside it in Figure 8(d) Instead of jump-ing abruptly between different representations the dis-appearance of the molecule and the appearance of atomsare gradual The transition between two representationsis more easily comprehensible Cognition required inunderstanding the change of visual image can be kept atthe level of perceptual processing so that users can freetheir cognitive processing capacities for more compli-cated tasks (Robertson Card amp Mackinlay 1991)

Scale-based semantic representations can be seen asone member of the family of level of detail (LOD) tech-

niques which visualize the same structure with differentgeometric objects according to a particular interactionparameter but with a very different purpose Whilemost LOD techniques such as distance LOD primarilyconcern computation efficiency (Puppo amp Scopigno1997) scale-based semantic representation gives moreconsideration to the userrsquos interaction needs CommonLOD techniques render individual objects with differenttextures or geometries but scale-based semantic repre-sentation visualizes structures with different semanticabstractions LOD techniques can be integrated intoscale-based semantic representations to improve bothuser interaction and machine performance For examplein a city-planning project a city can have different struc-tures at the level of city district and street to informplanners semantically and distance LOD can be appliedin rendering individual objects in each structure to im-prove 3D computation

332 Dynamic View to Support Cross-ScaleInformation Sharing Design here addresses issuesconcerning how to help users at different scales to buildcommon understandings of their divergent views Onedesign choice is to use a dynamic view to narrow downthe view difference We used animation to show thetransition between two views Animation is generatedby interpolating the view positions view orientationsand view scales of two views For example if two usersrsquoviews at two different scales look like Figures 7(a) and7(b) an animation can be generated by adding interme-diate image frames between Figure 7(a) and 7(b) Theanimation can tell how one view can be transformed toanother and how objects in two views are related toeach other

333 Direct Avatar Manipulation to SupportCross-Scale Action To help users get involved inothersrsquo work so that they can take advantage of theirdifferent action capabilities we adopted a design choicethat allows direct manipulation on other usersrsquo avatarsWith such a tool planners in the above city-planningexample can easily cooperate on tasks that require largeaction domain and high action accuracy Of course thisdirect interference should be regulated The user who

Figure 8 Successive fading views in semantic representations

Zhang and Furnas 39

owns the avatar is always aware of such action and hascontrol over whether this action is allowed

The implication of such a direct manipulation of ava-tar body for user interface design deserves more atten-tion Under this design an avatar can ldquoaffordrdquo manipu-lation activities This is quite different from thetraditional role of an avatar merely as user embodimentin conventional CVEs The reason that no direct manip-ulation of an avatar is allowed in conventional CVEsmight be due to the influence of real life where the au-tonomy of human bodies is well respected To betterfacilitate peoplersquos work in virtual environments somereal-life constraints could be relaxed or even abandonedVirtual environments should not be treated as the repli-cation of the real world just as new technologies shouldnot be interpreted by old metaphors Direct manipula-tion of a userrsquos avatar by others is an effort to breakdown the metaphor of reality

4 System Implementation

A desktop mCVE prototype system consisting ofover 20000 lines of Java code was implemented Dis-cussions in this section focus on the choice of toolkits insoftware implementation general system architectureand the implementation of some multiscale tools

41 Choice of Toolkits

The design of virtual environments that supportcollaboration needs to consider issues in three aspects3D graphics rendering network communication man-agement and user interactions (Singhal amp Zyda 1999)It is not our interest to address issues related to low-level graphics rendering or network communicationsOur goal here is to support user interactions with multi-scale structures and with other users at different scalesTo achieve this interaction-oriented design goal weimplemented the mCVE prototype with two toolkitsJava 3D and Java Shared Data Toolkit (JSDT)

This hybrid approach allows a design focus on high-level interaction tools because Java 3D can handlegraphics rendering and JSDT can handle network com-

munications issues With the scene-graph managementof Java 3D we can construct virtual environments andstructures at the level of geometric objects rather thanat the level of vertices and polygons Java 3D allows theextension of new rendering behaviors which is not wellsupported by other CVE toolkits such as DIVE (Carls-son amp Hagsand 1993) and SPLINE (Waters et al1997) This flexibility is critical to the implementationof such tools as scale-based semantic representations anddynamic view transition Furthermore the easy integra-tion of other Java APIs such as Java Swing into Java3D can simplify user interface design JSDT is ideal forthe research because it provides flexible data deliveryservices in support of synchronous collaboration on theInternet JSDT supports basic abstraction of networksessions multicast message communications and con-currency control Thus the design can focus on opti-mizing the management of distributed 3D scenes andcross-scale communications One drawback of this hy-brid approach is the lack of those CVE services seen inSPLINE (Waters et al) Collaboration services have tobe built from scratch

42 System Layer and Architecture

The system layer of the prototype is shown in Fig-ure 9 The implementation focused on the two layersinside the dashed rectangle On top of Java 3D andJSDT are multiscale graphics services and collaboration

Figure 9 System layer of the mCVE prototype

40 PRESENCE VOLUME 14 NUMBER 1

services and above them are multiscale collaborationtools

Multiscale graphics services handle object-renderingissues and update user views based on their view posi-tions and interaction scales Collaboration services de-liver messages among distributed users and manage theconsistency of shared virtual environments Multiscalecollaboration services deal with such collaboration issuesas cross-scale information sharing and cross-scale avatarmanipulation These services lay the foundation for multi-scale tools

43 Implementation of Multiscale Tools

A set of multiscale collaboration tools were imple-mented and integrated into the prototype system Thissection introduces such tools as scaling control whichdistinguishes mCVEs from other CVEs scale-based se-mantic representations which visualize information in amultiscale way cross-scale dynamic view transitionwhich addresses the cross-scale information problemuniquely seen in multiscale collaboration and the man-agement of avatar position in direct avatar manipulationwhich illustrates cross-scale data coordination

431 Scene Graph of the Prototype Systemand Scaling Control The scene graph of the systemis shown in Figure 10 Each Java 3D scene graph has aVirtual Universe object as the root A Locale object at-taches scene graphs to the Virtual Universe objectBranchGroup (BG) nodes collect such objects as geom-etry behavior light sound and so forth A Transform-Group (TG) node defines the coordinates of objectsunder it

The system has two subscene graphs the object scenegraph which is the left child of the Locale object andthe view scene graph which is the right child The ob-ject scene graph collects all objects and behavior nodesThe view scene graph defines the viewing behaviors

To change a userrsquos interaction scale is to change thevalues of the userrsquos interaction parameters relative to thesize of the virtual world Thus scale change can be im-plemented in two ways (1) scaling the virtual world upand down and keeping all interaction parameters un-

touched or (2) changing all interaction parameters andkeeping the virtual world untouched Mathematicallythey are equivalent We adopted the first approach be-cause scaling the virtual world up and down can bedone by modifying just one parameter the scaling fac-tor Si of the top TG in the object scene graph This ismuch simpler than the second approach which involvesthe modification of complicated view parameters

432 Scale-Based Semantic RepresentationsIn the implementation of scale-based semantic represen-tations we adopted a design that uses a delegate objectto wrap all representations and that delivers appropriaterepresentations based on the value of the controllingvariable (Fox 1998) A scale-based semantic representa-tion node is extended from the Java 3D Switch objectEach node has a set of child representations and an or-dered array that defines the scale range for each repre-sentation (Figure 11)

433 Animating View Transition in Collabo-rative View Sharing The key issue in making a view-transition animation is to interpolate two views InmCVEs a view can be uniquely determined by its viewposition P its view orientation O and its view scale Sand be written as V(P O S)

Given two views V0 V(P0 O 0 S0) and V1 V(P1O 1 S1) to implement a view-transition animation is to

Figure 10 Scene graph of the prototype system

Zhang and Furnas 41

choose a path that links V0 and V1 and to set the trajec-tory of viewpoint movement view direction along thepath and observation scale along the path View-pathand view-orientation interpolation techniques are ma-ture (Parent 2001 Watt amp Policarpo 2001) so thisresearch focused only on the interpolation of view scale

A scale-interpolation function could be linear loga-rithmic or in other forms We chose a logarithmic func-tion because it gives users a constant relative rate ofchange in their views Interpolating the view scale loga-rithmically can give users a familiar experience

Assume the animation includes n frames from V0 toV1 The view scale of the ith frame (0 i n) in a log-arithmic interpolation function can be written as

Si Si S0 (1)

where

S S1

S01n

(2)

With corresponding P i and O i from interpolationfunctions for view position and view orientation theview of the ith frame Vi V(P i O i Si) can be ob-tained Figure 12 shows two views seen in Figure 7 asFrame 0 and Frame n and other interpolated viewframes (Due to space limitations we have shown onlyfour frames used for the animation) Showing theseframes successively with a reasonable frame rate can pro-duce an animation linking these two divergent views

434 Cross-Scale Data Consistency and Ava-tar Position in Direct Avatar Manipulation Al-though changing the scaling factor of the top TG in theobject scene graph simplifies the scaling operation such

an approach poses a challenge in cross-scale data consis-tency In CVEs all users usually need to see the sameobject appearing at the same spatial location In conven-tional CVEs this can be easily achieved because an ob-jectrsquos coordinates in different usersrsquo worlds are often thesame In mCVEs however when users are at differentscales their virtual worlds are rescaled according to thescaling factors of the top TG in their own object scenegraphs The same object will have different local coordi-nates in different usersrsquo worlds When the object ismoved by a user the objectrsquos local coordinates in thisuserrsquos world cannot be used to update the objectrsquos newposition in all other usersrsquo worlds

In the implementation we used normalized coordi-nates which are calculated as the ratio of the object lo-cal coordinates in the userrsquos world to the userrsquos scaleLet Pl(xl yl zl) and Sl denote the local coordinates andthe userrsquos scale respectively The normalized coordi-nates Pn can be written as

Pn PlSl (3)

By doing so the same object will always have the samePn This Pn is used to define the coordinates of the ob-ject in every userrsquos object scene graph Modifying thespatial location of an object in a userrsquos world leads to anew Pn All users can use this Pn to update the coordi-nates of the object in their object scene graphs There-fore Pn enforces data consistency across scale Becausethe scaling factor is on the top of the scene graph theobjectrsquos final location in a userrsquos world is Pl the productof Pn and this userrsquos scale Sl With different Sl users willbe able to see the world rendered at different scales

An avatarrsquos position can be managed similarly butwith a small difference An avatar object differs fromother ordinary objects in a way that in its host userrsquos

Figure 11 Scale-based semantic representation node

Figure 12 Frames of a dynamic view-transition animation

42 PRESENCE VOLUME 14 NUMBER 1

world the coordinates of the avatar are affiliated withthe view scene graph rather than the object scenegraph so that the avatar always moves with the view Inother usersrsquo worlds however the avatar object is underthe object scene graphs Figure 13(a) is the scene graphof the world of a user say A who interacts with the vir-tual environment at a location Pa and with a scale SaFigure 13(b) is the scene graph of the world of anotheruser say B User Arsquos avatar is in User Brsquos object graphBecause Pa is not governed by Sa the normalized coor-dinates of the avatar in User Brsquos object graph should bewritten as

PnPaSa (4)

The body size of User Arsquos avatar changes with thescale of User A so a scale parameter is also needed todefine the size of User Arsquos avatar in User Brsquos objectscene graph so that User B can be aware of User Arsquosscale This scale should also be normalized Let Sn de-note the normalized scale where Sn is simply the recip-rocal of Sa

Sn 1Sa (5)

This is because the avatar body changes inversely pro-portional to the scale factor Sa (shrinking the world isequivalent to growing the avatar body and growing theworld is equivalent to shrinking the avatar body)

The implementation of other cross-scale tools includ-ing direct avatar manipulation also uses Pn and Sn todefine the coordinates and size of an avatar entity Thisapproach guarantees the consistency of the avatar posi-tion and size in all usersrsquo views at different scales

5 Experimental Evaluation

We evaluated multiscale collaboration in subjecttests An experiment was designed to assess the benefitsusers can get from an mCVE in accomplishing tasks re-quiring cross-scale coordination

51 Experiment Design and Procedure

Subject tasks in the experiment involved searchingfor a ldquobombrdquo on a square ground plane (2000 2000m2) with a distinctly shaped building in each corner(square hexagon octagon and circle as seen fromabove) Each had a height of 12 m and a base of about6400 m2 On the ground behind each building was aunit cube (1 m3) containing a unique text name andsmiley face (about 05 05 m2) The bomb was insideone of these four cubes In the test subjects were placedin the middle of the square plane They had a defaulteye level of 168 m and a default moving step of 1 m inthe virtual environment The shape of the building thatthe bomb was nearby was known in advance and sub-jects needed to find that building locate the bomb boxand key in the name of the box to defuse the bomb

A 2 2 2 factorial design that has six treatmentstwo noncollaboration and four collaboration wasadopted (Table 1) For the two noncollaboration treat-ments VE is just a conventional 3D virtual environ-ment and M-VE is a VE enhanced by multiscale toolsspecifically it allows users to change their interactionscales eye level and speed Among four collaborationtreatments CVE a conventional collaborative virtualenvironment is the only one without multiscale toolsThe other three treatments all equipped with multiscale

Figure 13 Two scene graphs for the same avatar

Table 1 Six Treatments in a 2 2 2 Design

Noncollaboration Collaboration

Nonmultiscale VE CVEMultiscale M-VE NR

GUIDEMOVE

Zhang and Furnas 43

tools differ in the assignment of subject task roles (be-ing a giant or a normal person) and in the way subjectsaffect each otherrsquos work across scales In one treatmentthe roles of subjects are not predefined and subjects canchoose their own interaction scales as desired This envi-ronment is labeled as NR (No Role) The other twomCVEs both assign one subject to be a giant and theother a normal person so that subjects can change theirinteraction scales only within a limited range In onesuch environment the giant is permitted to move thepartner directly and this treatment is labeled as MOVEAnother condition allows the giant to guide the move-ment of the partner only verbally and is denoted asGUIDE

Recruited through email 24 students paired in 12groups participated in the experiment In noncollabora-tion treatments they worked on their own and in col-laboration treatments they communicated through anaudio channel The performances were measured bytask-completion time

52 Results

An ANOVA analysis of data from four treatments(VE M-VE CVE and MOVE) shows main effects ofboth collaboration (F170 1298 p 0001) andmultiscale (F170 7090 p 00001) Interaction isnot significant (F170 187 p 176) as seen in Fig-ure 14 Subjects performed best in MOVE where theycould take full advantage of multiscale and collabora-tion and they did worst in VE where there was no helpat all

Subjects used different strategies in the different con-ditions Without multiscale tools subjects had to goaround and count the number of sides of all four build-ings to find the target a very time-consuming processWith multiscale subjects could increase their sizes to seebuilding shapes from above and approach the bombquickly so the time required was be reduced signifi-cantly

The performance difference among the three mCVEsalso indicates the importance of different collaborationsupports In the NR treatment subjects had to assumetheir roles via an expensive negotiation process In the

GUIDE treatment the giant subject who could see thebuilding shape must coordinate with the partner aboutnavigation verbally through a costly grounding process(Clark amp Brennan 1991) In MOVE however the all-seeing giant could actually move the partner to the des-tination quickly and bypassed grounding processes

6 Discussion

In this paper we have introduced a novel interac-tive environment multiscale collaborative virtual envi-ronment which gives users the freedom to control theirinteraction scales in collaboration Multiscale collabora-tion can help better allocate scarce scale resources andcombine different expertise and action capabilities ofindividual users in their work with large structures Wehave also discussed the implications of multiscale tech-nology for user interaction and the design and develop-ment of a Java-based mCVE system Our subject testshave shown that multiscale collaboration can facilitatepeoplersquos work that requires cross-scale information andcollaboration

Research reported in this paper has largely focused ongeneral interaction issues in mCVEs Designs have beenlimited to generic tools to support multiscale collabora-tion and experimental study has focused on generaltasks We expect that multiscale collaboration tools canbe used to support peoplersquos specific work in the realworld To achieve this goal the design of multiscale

Figure 14 Time (in seconds) for the six treatments

44 PRESENCE VOLUME 14 NUMBER 1

tools should support domain-specific tasks One of ourongoing research projects focuses on studying how multi-scale collaboration can support research in materials sci-ence and engineering We are investigating the needs ofmaterials scientists for multiscale collaboration and willdesign and implement tools that support these needsThe effectiveness and usability of these tools will also beevaluated when they are ready

At the same time we are interested in behavior issuesin mCVEs Some important research questions concern-ing user behavior in mCVEs have not been studiedthoroughly It is still unclear when and how users canbenefit from collaborative object manipulation acrossscale whether the designed dynamic view-transitiontechnique can indeed improve cross-scale informationsharing what mechanism is needed to help users bettermanage cross-scale collaboration overheads and soforth In addition it has been found that multiscale alsobrings some new challenges in social interactions(Zhang amp Furnas 2002) and efforts are needed tostudy what can be done to address these interaction is-sues Research in this direction will deepen our under-standing of mCVEs and help to design better tools tosupport multiscale collaboration

Acknowledgments

This research was funded in part by Microsoft Research

References

Ahl V amp Allen T (1996) Hierarchy theory A vision vocab-ulary and epistemology New York Columbia UniversityPress

Alexander C Ishikawa S amp Silverstein M (1977) A pat-tern language Oxford UK Oxford University Press

Bederson B amp Hollan J (1994) Pad A zoominggraphical interface for exploring alternate interface physicsProceedings of ACM UIST rsquo94 17ndash26

Benford S Bowers J Fahlen L Greenhalgh C amp Snow-don D (1995) User embodiment in collaborative virtualenvironments Proceedings of ACM CHI rsquo95 242ndash249

Benford S Snowdon D Colebourne A OrsquoBrien J ampRodden T (1997) Informing the design of collaborativevirtual environments Proceedings of ACM GROUP rsquo97 71ndash80

Carlsson C amp Hagsand O (1993) DIVEmdashA platform formulti-user virtual environments Computer and Graphics17(6) 663ndash669

Clark H amp Brennan S (1991) Grounding in communica-tion In L B Resnick J Levine amp S D Behrend (Eds)Perspectives on socially shared cognition (pp 127ndash149)Washington DC American Psychological Association

Dix A J (1994) Computer-supported cooperative workmdashAframework In D Rosenburg amp C Hutchison (Eds) De-sign Issues in CSCW (pp 23ndash37) Berlin Springer-Verlag

EPFL Geneva IIS TNO Lancaster UCL et al (1997)Guidelines for building CVE applications COVEN ProjectRetrieved November 21 2002 from httpwwwcsuclacukstaffmslatervrCovenPublicDeliverablesdel26PDF

Fox D (1998) Tabula rasa A multi-scale user interface sys-tem Unpublished doctoral dissertation New York Univer-sity

Furnas G (1986) Generalized fisheye views Proceedings ofACM CHI rsquo86 16ndash23

Gaver W Sellen A Heath C amp Luff P (1993) One isnot enough Multiple views in a media space Proceedings ofACM INTERCHI rsquo93 335ndash341

Hollan J amp Stornetta S (1992) Beyond being there Pro-ceedings of ACM CHI rsquo92 119ndash125

Koike H amp Yoshihara H (1993) Fractal approaches forvisualizing huge hierarchies Proceedings of IEEE VL rsquo9355ndash60

Lamping J Rao R amp Pirolli P (1995) A focus contexttechnique based on hyperbolic geometry for visualizinglarge hierarchies Proceedings of ACM CHI rsquo95 401ndash408

Leigh J Johnson A amp DeFanti T (1996) CALVIN Animmersimedia design environment utilizing heterogeneousperspectives Proceedings of IEEE International Conferenceon Multimedia Computing and Systems 20ndash23

Lieberman H (1994) Powers of ten thousand Navigating inlarge information spaces Proceedings of ACM UIST rsquo9415ndash16

Mackinlay J Card S amp Robertson G (1990) Rapid con-trolled movement through a virtual 3D workspace Proceed-ings of ACM SIGGRAPH rsquo90 171ndash176

Mine M Brooks F amp Sequin C (1997) Moving objects

Zhang and Furnas 45

in space Exploiting proprioception in virtual-environmentinteraction Proceedings of ACM SIGGRAPH rsquo97 19ndash26

Morrison P amp Morrison P (1994) Powers of ten About therelative size of things in the universe and the effect of addinganother zero San Francisco W H Freeman

Parent R (2001) Computer animation Algorithms and tech-niques San Francisco Morgan-Kaufmann

Pattee H (1973) Hierarchy theory The challenge of complexsystems New York George Braziller

Perlin K amp Fox D (1993) Pad An alternative approach tothe computer interface Proceedings of ACM SIGGRAPHrsquo93 57ndash64

Pierce J Forsberg A Conway M Hong S Zeleznik Ramp Mine M (1997) Image plane interaction techniques in3D immersive environments Proceedings of the 1997 Sympo-sium on Interactive 3D Graphics 39ndash43

Poupyrev I Billinghurst M Weghorst S amp Ichikawa T(1996) The go-go interaction technique Non-linear map-ping for direct manipulation in VR Proceedings of ACMUIST rsquo96 79ndash80

Puppo E amp Scopigno R (1997) Simplification LOD multi-resolution Principles and applications Proceedings ofEUROGRAPHICS 16 (3)

Raab A amp Ruger M (1996) 3D-Zoom Interactive visual-ization of structures and relations in complex graphics InB Girod H Nieman amp H Seidel (Eds) 3D image analy-sis and synthesis (pp 125ndash132) Sankt Augustin GermanyInfix-Verlag

Robbins M (2001) Modeling and simulation of the failureand toughness of polymersubstrate bonds The second An-nual Multiscale ModelingmdashBridging the Gap Between Atom-istic and Meso Scales

Robertson G Card S amp Mackinlay J (1991) Informationvisualization using 3D interactive animation Proceedings ofACM CHI rsquo91 461ndash462

Robertson G amp Mackinlay J (1993) The document lensProceedings of ACM UIST rsquo93 101ndash110

Robertson G Mackinlay J amp Card S (1991) Cone treesAnimated 3D visualizations of hierarchical informationCommunications of the ACM 34 (2) p 89ndash194

Sarkar M amp Brown M H (1992) Graphical fisheye viewsof graphs Proceedings of ACM CHI rsquo92 83ndash91

Singhal S amp Zyda M (1999) Networked virtual environ-ments Design and implementation New York ACM Press

Snowdon D amp Jaa-Aro K (1997) A subjective virtual envi-ronment for collaborative information visualization VirtualReality Universe rsquo97 Retrieved May 23 2003 fromhttpwwwcyberedgecomvru_paperssnowdenhtm

Stappers P Gaver W amp Overbeeke C (2000) Beyond thelimits of real-time realism 2000 Chapter to appear in L JHettinger amp M W Haas (Eds) Psychological issues in thedesign and use of virtual and adaptive environment Mah-wah NJ Erlbaum

Stefik M Bobrow D Foster G Lanning S amp Tatar D(1987) WYSIWIS revised Early experiences with multiuserinterfaces ACM Transactions on Information Systems(TOIS) 5(2) 147ndash167

Stoakley R Conway M amp Pausch R (1995) Virtual real-ity on a WIM Interactive worlds in miniature Proceedingsof ACM CHI rsquo95 265ndash272

Sutherland I (1965) The ultimate display Proceedings ofIFIP Congress 2

Tromp J G (1999) Usability design for CVEs COVENProject Retrieved November 21 2002 from httpcovenlancsacuk4deliverablesdel29pdf

Waters R Anderson D Barrus J Brogan D Casey MMcKeown S et al (1997) Diamond park and SPLINESocial virtual reality with 3D animation spoken interactionand runtime extendibility Presence Teleoperators and Vir-tual Environments 6(4) 461ndash481

Watt A amp Policarpo F (2001) 3D games Animation andadvanced real-time rendering New York Addison-Wesley

Zhang X amp Furnas G (2002) Social interactions in multi-scale CVEs Proceedings of ACM CVE 2002 31ndash38

46 PRESENCE VOLUME 14 NUMBER 1

different perspectives and show different levels of ab-straction of the city may make it difficult for two plan-ners to understand how their work can be related Aplanner can refer to a particular object in her view butthe other may not see it at all Multiscale technologymay hurt mutual understanding of objects in collabora-tion because of scale-related views

313 Cross-Scale Action Multiscale can alsobring some benefits to collaboration One of them isthat users can leverage their different action capabilitiesand different scopes of influence of their actions Takethe example of the two city planners again When thestreet-level planner needs to go to a distant place thecity-level planner with a great action domain can movethe partner easily to the destination and reduce the tasktime Or when the city-level planner wants to move toan exact location the street-level planner can move thepartnerrsquos avatar accurately because of high manipulationaccuracy It is mentioned previously that a similar tech-nique can be used in object manipulation Thus multi-scale collaboration can improve navigation and manipu-lation activities by allowing users to take advantage ofeach otherrsquos different action capabilities

32 Design Considerations

Since multiscale technology could affect user in-teraction both positively and negatively the design ofmCVEs focuses on maximizing the advantages of multi-scale technology and avoiding the disadvantages Thisprinciple leads to the following design considerations

321 Support for Better Understanding ofMultiscale Structures Multiscale structures shouldbe modeled in such a way that scale-dependent charac-teristics can be observed at different scale levels In thereal world multiscale structures are ubiquitous as seenin Figures 1 and 2 To observe these multiscale struc-tures we need only appropriate tools In 3D virtualworlds however what users can see is predesignedThus to visualize multiscale structures objects have tobe modeled and rendered in a multiscale way

322 Support for Cross-Scale InformationSharing Users should be able to share artifacts ofcommon interest with others across scale Tools such asmultiple views of the world (Gaver Sellen Heath ampLuff 1993) cannot help too much Such techniquesrequire some common objects in different views as ref-erences but usersrsquo very subjective views in mCVEs maymake it difficult to find such references It is a challengeto build mutual understanding of working contexts withsubjective views (Snowdon amp Jaa-Aro 1997) Usersneed help in understanding how their subjective viewsand objects in them are related to each other

323 Support for Cross-Scale Action Usersshould be allowed to get involved in othersrsquo work acrossscale so that they can leverage their different interactioncapabilities in collaboration They should be able to di-rectly manipulate objects that others are working onand move the avatars of other users Of course this in-tervention should be based on mutual agreements toavoid any unwanted consequences In the city-planningexample the city-level planner should not change thingsthe street-level planner is working on without consentOtherwise the street-level planner may get confusedand not understand what has happened

33 Conceptual Design

Based on these considerations some multiscaletools were designed Here discussions focus on designsthat are fundamentally important to mCVEs includingscale-based semantic representations that present scale-related characteristics of multiscale structures dynamic

Figure 7 Two views of a city at two scale levels

38 PRESENCE VOLUME 14 NUMBER 1

view transition that facilitates cross-scale informationsharing and direct avatar manipulation that supportscross-scale action

331 Scale-Based Semantic Representationsfor Multiscale Structures This design renders amultiscale structure with different representations atdifferent scales Figure 5 presents the visual results of asubstance under this technique Initially the substanceappears as a double helix in (a) The user changes thescale to increase the structure size and to see more de-tails in (b) Continuing the scaling the user finds a dif-ferent structure in an atom in (c) The user is informedof semantic characteristics of the substance at differentscales

This scale-based representation technique has oneproblem The change between two very different repre-sentations may confuse users For example the suddenappearance of the structure in Figure 5(c) may distractusers from their primary tasks To address this issue weadopted a fading tool (Bederson amp Hollan 1994Lieberman 1994) to smooth the transition betweendifferent representations Figure 8 shows a transitionprocess from a spherelike molecule from Figure 8(a) toa set of atoms inside it in Figure 8(d) Instead of jump-ing abruptly between different representations the dis-appearance of the molecule and the appearance of atomsare gradual The transition between two representationsis more easily comprehensible Cognition required inunderstanding the change of visual image can be kept atthe level of perceptual processing so that users can freetheir cognitive processing capacities for more compli-cated tasks (Robertson Card amp Mackinlay 1991)

Scale-based semantic representations can be seen asone member of the family of level of detail (LOD) tech-

niques which visualize the same structure with differentgeometric objects according to a particular interactionparameter but with a very different purpose Whilemost LOD techniques such as distance LOD primarilyconcern computation efficiency (Puppo amp Scopigno1997) scale-based semantic representation gives moreconsideration to the userrsquos interaction needs CommonLOD techniques render individual objects with differenttextures or geometries but scale-based semantic repre-sentation visualizes structures with different semanticabstractions LOD techniques can be integrated intoscale-based semantic representations to improve bothuser interaction and machine performance For examplein a city-planning project a city can have different struc-tures at the level of city district and street to informplanners semantically and distance LOD can be appliedin rendering individual objects in each structure to im-prove 3D computation

332 Dynamic View to Support Cross-ScaleInformation Sharing Design here addresses issuesconcerning how to help users at different scales to buildcommon understandings of their divergent views Onedesign choice is to use a dynamic view to narrow downthe view difference We used animation to show thetransition between two views Animation is generatedby interpolating the view positions view orientationsand view scales of two views For example if two usersrsquoviews at two different scales look like Figures 7(a) and7(b) an animation can be generated by adding interme-diate image frames between Figure 7(a) and 7(b) Theanimation can tell how one view can be transformed toanother and how objects in two views are related toeach other

333 Direct Avatar Manipulation to SupportCross-Scale Action To help users get involved inothersrsquo work so that they can take advantage of theirdifferent action capabilities we adopted a design choicethat allows direct manipulation on other usersrsquo avatarsWith such a tool planners in the above city-planningexample can easily cooperate on tasks that require largeaction domain and high action accuracy Of course thisdirect interference should be regulated The user who

Figure 8 Successive fading views in semantic representations

Zhang and Furnas 39

owns the avatar is always aware of such action and hascontrol over whether this action is allowed

The implication of such a direct manipulation of ava-tar body for user interface design deserves more atten-tion Under this design an avatar can ldquoaffordrdquo manipu-lation activities This is quite different from thetraditional role of an avatar merely as user embodimentin conventional CVEs The reason that no direct manip-ulation of an avatar is allowed in conventional CVEsmight be due to the influence of real life where the au-tonomy of human bodies is well respected To betterfacilitate peoplersquos work in virtual environments somereal-life constraints could be relaxed or even abandonedVirtual environments should not be treated as the repli-cation of the real world just as new technologies shouldnot be interpreted by old metaphors Direct manipula-tion of a userrsquos avatar by others is an effort to breakdown the metaphor of reality

4 System Implementation

A desktop mCVE prototype system consisting ofover 20000 lines of Java code was implemented Dis-cussions in this section focus on the choice of toolkits insoftware implementation general system architectureand the implementation of some multiscale tools

41 Choice of Toolkits

The design of virtual environments that supportcollaboration needs to consider issues in three aspects3D graphics rendering network communication man-agement and user interactions (Singhal amp Zyda 1999)It is not our interest to address issues related to low-level graphics rendering or network communicationsOur goal here is to support user interactions with multi-scale structures and with other users at different scalesTo achieve this interaction-oriented design goal weimplemented the mCVE prototype with two toolkitsJava 3D and Java Shared Data Toolkit (JSDT)

This hybrid approach allows a design focus on high-level interaction tools because Java 3D can handlegraphics rendering and JSDT can handle network com-

munications issues With the scene-graph managementof Java 3D we can construct virtual environments andstructures at the level of geometric objects rather thanat the level of vertices and polygons Java 3D allows theextension of new rendering behaviors which is not wellsupported by other CVE toolkits such as DIVE (Carls-son amp Hagsand 1993) and SPLINE (Waters et al1997) This flexibility is critical to the implementationof such tools as scale-based semantic representations anddynamic view transition Furthermore the easy integra-tion of other Java APIs such as Java Swing into Java3D can simplify user interface design JSDT is ideal forthe research because it provides flexible data deliveryservices in support of synchronous collaboration on theInternet JSDT supports basic abstraction of networksessions multicast message communications and con-currency control Thus the design can focus on opti-mizing the management of distributed 3D scenes andcross-scale communications One drawback of this hy-brid approach is the lack of those CVE services seen inSPLINE (Waters et al) Collaboration services have tobe built from scratch

42 System Layer and Architecture

The system layer of the prototype is shown in Fig-ure 9 The implementation focused on the two layersinside the dashed rectangle On top of Java 3D andJSDT are multiscale graphics services and collaboration

Figure 9 System layer of the mCVE prototype

40 PRESENCE VOLUME 14 NUMBER 1

services and above them are multiscale collaborationtools

Multiscale graphics services handle object-renderingissues and update user views based on their view posi-tions and interaction scales Collaboration services de-liver messages among distributed users and manage theconsistency of shared virtual environments Multiscalecollaboration services deal with such collaboration issuesas cross-scale information sharing and cross-scale avatarmanipulation These services lay the foundation for multi-scale tools

43 Implementation of Multiscale Tools

A set of multiscale collaboration tools were imple-mented and integrated into the prototype system Thissection introduces such tools as scaling control whichdistinguishes mCVEs from other CVEs scale-based se-mantic representations which visualize information in amultiscale way cross-scale dynamic view transitionwhich addresses the cross-scale information problemuniquely seen in multiscale collaboration and the man-agement of avatar position in direct avatar manipulationwhich illustrates cross-scale data coordination

431 Scene Graph of the Prototype Systemand Scaling Control The scene graph of the systemis shown in Figure 10 Each Java 3D scene graph has aVirtual Universe object as the root A Locale object at-taches scene graphs to the Virtual Universe objectBranchGroup (BG) nodes collect such objects as geom-etry behavior light sound and so forth A Transform-Group (TG) node defines the coordinates of objectsunder it

The system has two subscene graphs the object scenegraph which is the left child of the Locale object andthe view scene graph which is the right child The ob-ject scene graph collects all objects and behavior nodesThe view scene graph defines the viewing behaviors

To change a userrsquos interaction scale is to change thevalues of the userrsquos interaction parameters relative to thesize of the virtual world Thus scale change can be im-plemented in two ways (1) scaling the virtual world upand down and keeping all interaction parameters un-

touched or (2) changing all interaction parameters andkeeping the virtual world untouched Mathematicallythey are equivalent We adopted the first approach be-cause scaling the virtual world up and down can bedone by modifying just one parameter the scaling fac-tor Si of the top TG in the object scene graph This ismuch simpler than the second approach which involvesthe modification of complicated view parameters

432 Scale-Based Semantic RepresentationsIn the implementation of scale-based semantic represen-tations we adopted a design that uses a delegate objectto wrap all representations and that delivers appropriaterepresentations based on the value of the controllingvariable (Fox 1998) A scale-based semantic representa-tion node is extended from the Java 3D Switch objectEach node has a set of child representations and an or-dered array that defines the scale range for each repre-sentation (Figure 11)

433 Animating View Transition in Collabo-rative View Sharing The key issue in making a view-transition animation is to interpolate two views InmCVEs a view can be uniquely determined by its viewposition P its view orientation O and its view scale Sand be written as V(P O S)

Given two views V0 V(P0 O 0 S0) and V1 V(P1O 1 S1) to implement a view-transition animation is to

Figure 10 Scene graph of the prototype system

Zhang and Furnas 41

choose a path that links V0 and V1 and to set the trajec-tory of viewpoint movement view direction along thepath and observation scale along the path View-pathand view-orientation interpolation techniques are ma-ture (Parent 2001 Watt amp Policarpo 2001) so thisresearch focused only on the interpolation of view scale

A scale-interpolation function could be linear loga-rithmic or in other forms We chose a logarithmic func-tion because it gives users a constant relative rate ofchange in their views Interpolating the view scale loga-rithmically can give users a familiar experience

Assume the animation includes n frames from V0 toV1 The view scale of the ith frame (0 i n) in a log-arithmic interpolation function can be written as

Si Si S0 (1)

where

S S1

S01n

(2)

With corresponding P i and O i from interpolationfunctions for view position and view orientation theview of the ith frame Vi V(P i O i Si) can be ob-tained Figure 12 shows two views seen in Figure 7 asFrame 0 and Frame n and other interpolated viewframes (Due to space limitations we have shown onlyfour frames used for the animation) Showing theseframes successively with a reasonable frame rate can pro-duce an animation linking these two divergent views

434 Cross-Scale Data Consistency and Ava-tar Position in Direct Avatar Manipulation Al-though changing the scaling factor of the top TG in theobject scene graph simplifies the scaling operation such

an approach poses a challenge in cross-scale data consis-tency In CVEs all users usually need to see the sameobject appearing at the same spatial location In conven-tional CVEs this can be easily achieved because an ob-jectrsquos coordinates in different usersrsquo worlds are often thesame In mCVEs however when users are at differentscales their virtual worlds are rescaled according to thescaling factors of the top TG in their own object scenegraphs The same object will have different local coordi-nates in different usersrsquo worlds When the object ismoved by a user the objectrsquos local coordinates in thisuserrsquos world cannot be used to update the objectrsquos newposition in all other usersrsquo worlds

In the implementation we used normalized coordi-nates which are calculated as the ratio of the object lo-cal coordinates in the userrsquos world to the userrsquos scaleLet Pl(xl yl zl) and Sl denote the local coordinates andthe userrsquos scale respectively The normalized coordi-nates Pn can be written as

Pn PlSl (3)

By doing so the same object will always have the samePn This Pn is used to define the coordinates of the ob-ject in every userrsquos object scene graph Modifying thespatial location of an object in a userrsquos world leads to anew Pn All users can use this Pn to update the coordi-nates of the object in their object scene graphs There-fore Pn enforces data consistency across scale Becausethe scaling factor is on the top of the scene graph theobjectrsquos final location in a userrsquos world is Pl the productof Pn and this userrsquos scale Sl With different Sl users willbe able to see the world rendered at different scales

An avatarrsquos position can be managed similarly butwith a small difference An avatar object differs fromother ordinary objects in a way that in its host userrsquos

Figure 11 Scale-based semantic representation node

Figure 12 Frames of a dynamic view-transition animation

42 PRESENCE VOLUME 14 NUMBER 1

world the coordinates of the avatar are affiliated withthe view scene graph rather than the object scenegraph so that the avatar always moves with the view Inother usersrsquo worlds however the avatar object is underthe object scene graphs Figure 13(a) is the scene graphof the world of a user say A who interacts with the vir-tual environment at a location Pa and with a scale SaFigure 13(b) is the scene graph of the world of anotheruser say B User Arsquos avatar is in User Brsquos object graphBecause Pa is not governed by Sa the normalized coor-dinates of the avatar in User Brsquos object graph should bewritten as

PnPaSa (4)

The body size of User Arsquos avatar changes with thescale of User A so a scale parameter is also needed todefine the size of User Arsquos avatar in User Brsquos objectscene graph so that User B can be aware of User Arsquosscale This scale should also be normalized Let Sn de-note the normalized scale where Sn is simply the recip-rocal of Sa

Sn 1Sa (5)

This is because the avatar body changes inversely pro-portional to the scale factor Sa (shrinking the world isequivalent to growing the avatar body and growing theworld is equivalent to shrinking the avatar body)

The implementation of other cross-scale tools includ-ing direct avatar manipulation also uses Pn and Sn todefine the coordinates and size of an avatar entity Thisapproach guarantees the consistency of the avatar posi-tion and size in all usersrsquo views at different scales

5 Experimental Evaluation

We evaluated multiscale collaboration in subjecttests An experiment was designed to assess the benefitsusers can get from an mCVE in accomplishing tasks re-quiring cross-scale coordination

51 Experiment Design and Procedure

Subject tasks in the experiment involved searchingfor a ldquobombrdquo on a square ground plane (2000 2000m2) with a distinctly shaped building in each corner(square hexagon octagon and circle as seen fromabove) Each had a height of 12 m and a base of about6400 m2 On the ground behind each building was aunit cube (1 m3) containing a unique text name andsmiley face (about 05 05 m2) The bomb was insideone of these four cubes In the test subjects were placedin the middle of the square plane They had a defaulteye level of 168 m and a default moving step of 1 m inthe virtual environment The shape of the building thatthe bomb was nearby was known in advance and sub-jects needed to find that building locate the bomb boxand key in the name of the box to defuse the bomb

A 2 2 2 factorial design that has six treatmentstwo noncollaboration and four collaboration wasadopted (Table 1) For the two noncollaboration treat-ments VE is just a conventional 3D virtual environ-ment and M-VE is a VE enhanced by multiscale toolsspecifically it allows users to change their interactionscales eye level and speed Among four collaborationtreatments CVE a conventional collaborative virtualenvironment is the only one without multiscale toolsThe other three treatments all equipped with multiscale

Figure 13 Two scene graphs for the same avatar

Table 1 Six Treatments in a 2 2 2 Design

Noncollaboration Collaboration

Nonmultiscale VE CVEMultiscale M-VE NR

GUIDEMOVE

Zhang and Furnas 43

tools differ in the assignment of subject task roles (be-ing a giant or a normal person) and in the way subjectsaffect each otherrsquos work across scales In one treatmentthe roles of subjects are not predefined and subjects canchoose their own interaction scales as desired This envi-ronment is labeled as NR (No Role) The other twomCVEs both assign one subject to be a giant and theother a normal person so that subjects can change theirinteraction scales only within a limited range In onesuch environment the giant is permitted to move thepartner directly and this treatment is labeled as MOVEAnother condition allows the giant to guide the move-ment of the partner only verbally and is denoted asGUIDE

Recruited through email 24 students paired in 12groups participated in the experiment In noncollabora-tion treatments they worked on their own and in col-laboration treatments they communicated through anaudio channel The performances were measured bytask-completion time

52 Results

An ANOVA analysis of data from four treatments(VE M-VE CVE and MOVE) shows main effects ofboth collaboration (F170 1298 p 0001) andmultiscale (F170 7090 p 00001) Interaction isnot significant (F170 187 p 176) as seen in Fig-ure 14 Subjects performed best in MOVE where theycould take full advantage of multiscale and collabora-tion and they did worst in VE where there was no helpat all

Subjects used different strategies in the different con-ditions Without multiscale tools subjects had to goaround and count the number of sides of all four build-ings to find the target a very time-consuming processWith multiscale subjects could increase their sizes to seebuilding shapes from above and approach the bombquickly so the time required was be reduced signifi-cantly

The performance difference among the three mCVEsalso indicates the importance of different collaborationsupports In the NR treatment subjects had to assumetheir roles via an expensive negotiation process In the

GUIDE treatment the giant subject who could see thebuilding shape must coordinate with the partner aboutnavigation verbally through a costly grounding process(Clark amp Brennan 1991) In MOVE however the all-seeing giant could actually move the partner to the des-tination quickly and bypassed grounding processes

6 Discussion

In this paper we have introduced a novel interac-tive environment multiscale collaborative virtual envi-ronment which gives users the freedom to control theirinteraction scales in collaboration Multiscale collabora-tion can help better allocate scarce scale resources andcombine different expertise and action capabilities ofindividual users in their work with large structures Wehave also discussed the implications of multiscale tech-nology for user interaction and the design and develop-ment of a Java-based mCVE system Our subject testshave shown that multiscale collaboration can facilitatepeoplersquos work that requires cross-scale information andcollaboration

Research reported in this paper has largely focused ongeneral interaction issues in mCVEs Designs have beenlimited to generic tools to support multiscale collabora-tion and experimental study has focused on generaltasks We expect that multiscale collaboration tools canbe used to support peoplersquos specific work in the realworld To achieve this goal the design of multiscale

Figure 14 Time (in seconds) for the six treatments

44 PRESENCE VOLUME 14 NUMBER 1

tools should support domain-specific tasks One of ourongoing research projects focuses on studying how multi-scale collaboration can support research in materials sci-ence and engineering We are investigating the needs ofmaterials scientists for multiscale collaboration and willdesign and implement tools that support these needsThe effectiveness and usability of these tools will also beevaluated when they are ready

At the same time we are interested in behavior issuesin mCVEs Some important research questions concern-ing user behavior in mCVEs have not been studiedthoroughly It is still unclear when and how users canbenefit from collaborative object manipulation acrossscale whether the designed dynamic view-transitiontechnique can indeed improve cross-scale informationsharing what mechanism is needed to help users bettermanage cross-scale collaboration overheads and soforth In addition it has been found that multiscale alsobrings some new challenges in social interactions(Zhang amp Furnas 2002) and efforts are needed tostudy what can be done to address these interaction is-sues Research in this direction will deepen our under-standing of mCVEs and help to design better tools tosupport multiscale collaboration

Acknowledgments

This research was funded in part by Microsoft Research

References

Ahl V amp Allen T (1996) Hierarchy theory A vision vocab-ulary and epistemology New York Columbia UniversityPress

Alexander C Ishikawa S amp Silverstein M (1977) A pat-tern language Oxford UK Oxford University Press

Bederson B amp Hollan J (1994) Pad A zoominggraphical interface for exploring alternate interface physicsProceedings of ACM UIST rsquo94 17ndash26

Benford S Bowers J Fahlen L Greenhalgh C amp Snow-don D (1995) User embodiment in collaborative virtualenvironments Proceedings of ACM CHI rsquo95 242ndash249

Benford S Snowdon D Colebourne A OrsquoBrien J ampRodden T (1997) Informing the design of collaborativevirtual environments Proceedings of ACM GROUP rsquo97 71ndash80

Carlsson C amp Hagsand O (1993) DIVEmdashA platform formulti-user virtual environments Computer and Graphics17(6) 663ndash669

Clark H amp Brennan S (1991) Grounding in communica-tion In L B Resnick J Levine amp S D Behrend (Eds)Perspectives on socially shared cognition (pp 127ndash149)Washington DC American Psychological Association

Dix A J (1994) Computer-supported cooperative workmdashAframework In D Rosenburg amp C Hutchison (Eds) De-sign Issues in CSCW (pp 23ndash37) Berlin Springer-Verlag

EPFL Geneva IIS TNO Lancaster UCL et al (1997)Guidelines for building CVE applications COVEN ProjectRetrieved November 21 2002 from httpwwwcsuclacukstaffmslatervrCovenPublicDeliverablesdel26PDF

Fox D (1998) Tabula rasa A multi-scale user interface sys-tem Unpublished doctoral dissertation New York Univer-sity

Furnas G (1986) Generalized fisheye views Proceedings ofACM CHI rsquo86 16ndash23

Gaver W Sellen A Heath C amp Luff P (1993) One isnot enough Multiple views in a media space Proceedings ofACM INTERCHI rsquo93 335ndash341

Hollan J amp Stornetta S (1992) Beyond being there Pro-ceedings of ACM CHI rsquo92 119ndash125

Koike H amp Yoshihara H (1993) Fractal approaches forvisualizing huge hierarchies Proceedings of IEEE VL rsquo9355ndash60

Lamping J Rao R amp Pirolli P (1995) A focus contexttechnique based on hyperbolic geometry for visualizinglarge hierarchies Proceedings of ACM CHI rsquo95 401ndash408

Leigh J Johnson A amp DeFanti T (1996) CALVIN Animmersimedia design environment utilizing heterogeneousperspectives Proceedings of IEEE International Conferenceon Multimedia Computing and Systems 20ndash23

Lieberman H (1994) Powers of ten thousand Navigating inlarge information spaces Proceedings of ACM UIST rsquo9415ndash16

Mackinlay J Card S amp Robertson G (1990) Rapid con-trolled movement through a virtual 3D workspace Proceed-ings of ACM SIGGRAPH rsquo90 171ndash176

Mine M Brooks F amp Sequin C (1997) Moving objects

Zhang and Furnas 45

in space Exploiting proprioception in virtual-environmentinteraction Proceedings of ACM SIGGRAPH rsquo97 19ndash26

Morrison P amp Morrison P (1994) Powers of ten About therelative size of things in the universe and the effect of addinganother zero San Francisco W H Freeman

Parent R (2001) Computer animation Algorithms and tech-niques San Francisco Morgan-Kaufmann

Pattee H (1973) Hierarchy theory The challenge of complexsystems New York George Braziller

Perlin K amp Fox D (1993) Pad An alternative approach tothe computer interface Proceedings of ACM SIGGRAPHrsquo93 57ndash64

Pierce J Forsberg A Conway M Hong S Zeleznik Ramp Mine M (1997) Image plane interaction techniques in3D immersive environments Proceedings of the 1997 Sympo-sium on Interactive 3D Graphics 39ndash43

Poupyrev I Billinghurst M Weghorst S amp Ichikawa T(1996) The go-go interaction technique Non-linear map-ping for direct manipulation in VR Proceedings of ACMUIST rsquo96 79ndash80

Puppo E amp Scopigno R (1997) Simplification LOD multi-resolution Principles and applications Proceedings ofEUROGRAPHICS 16 (3)

Raab A amp Ruger M (1996) 3D-Zoom Interactive visual-ization of structures and relations in complex graphics InB Girod H Nieman amp H Seidel (Eds) 3D image analy-sis and synthesis (pp 125ndash132) Sankt Augustin GermanyInfix-Verlag

Robbins M (2001) Modeling and simulation of the failureand toughness of polymersubstrate bonds The second An-nual Multiscale ModelingmdashBridging the Gap Between Atom-istic and Meso Scales

Robertson G Card S amp Mackinlay J (1991) Informationvisualization using 3D interactive animation Proceedings ofACM CHI rsquo91 461ndash462

Robertson G amp Mackinlay J (1993) The document lensProceedings of ACM UIST rsquo93 101ndash110

Robertson G Mackinlay J amp Card S (1991) Cone treesAnimated 3D visualizations of hierarchical informationCommunications of the ACM 34 (2) p 89ndash194

Sarkar M amp Brown M H (1992) Graphical fisheye viewsof graphs Proceedings of ACM CHI rsquo92 83ndash91

Singhal S amp Zyda M (1999) Networked virtual environ-ments Design and implementation New York ACM Press

Snowdon D amp Jaa-Aro K (1997) A subjective virtual envi-ronment for collaborative information visualization VirtualReality Universe rsquo97 Retrieved May 23 2003 fromhttpwwwcyberedgecomvru_paperssnowdenhtm

Stappers P Gaver W amp Overbeeke C (2000) Beyond thelimits of real-time realism 2000 Chapter to appear in L JHettinger amp M W Haas (Eds) Psychological issues in thedesign and use of virtual and adaptive environment Mah-wah NJ Erlbaum

Stefik M Bobrow D Foster G Lanning S amp Tatar D(1987) WYSIWIS revised Early experiences with multiuserinterfaces ACM Transactions on Information Systems(TOIS) 5(2) 147ndash167

Stoakley R Conway M amp Pausch R (1995) Virtual real-ity on a WIM Interactive worlds in miniature Proceedingsof ACM CHI rsquo95 265ndash272

Sutherland I (1965) The ultimate display Proceedings ofIFIP Congress 2

Tromp J G (1999) Usability design for CVEs COVENProject Retrieved November 21 2002 from httpcovenlancsacuk4deliverablesdel29pdf

Waters R Anderson D Barrus J Brogan D Casey MMcKeown S et al (1997) Diamond park and SPLINESocial virtual reality with 3D animation spoken interactionand runtime extendibility Presence Teleoperators and Vir-tual Environments 6(4) 461ndash481

Watt A amp Policarpo F (2001) 3D games Animation andadvanced real-time rendering New York Addison-Wesley

Zhang X amp Furnas G (2002) Social interactions in multi-scale CVEs Proceedings of ACM CVE 2002 31ndash38

46 PRESENCE VOLUME 14 NUMBER 1

view transition that facilitates cross-scale informationsharing and direct avatar manipulation that supportscross-scale action

331 Scale-Based Semantic Representationsfor Multiscale Structures This design renders amultiscale structure with different representations atdifferent scales Figure 5 presents the visual results of asubstance under this technique Initially the substanceappears as a double helix in (a) The user changes thescale to increase the structure size and to see more de-tails in (b) Continuing the scaling the user finds a dif-ferent structure in an atom in (c) The user is informedof semantic characteristics of the substance at differentscales

This scale-based representation technique has oneproblem The change between two very different repre-sentations may confuse users For example the suddenappearance of the structure in Figure 5(c) may distractusers from their primary tasks To address this issue weadopted a fading tool (Bederson amp Hollan 1994Lieberman 1994) to smooth the transition betweendifferent representations Figure 8 shows a transitionprocess from a spherelike molecule from Figure 8(a) toa set of atoms inside it in Figure 8(d) Instead of jump-ing abruptly between different representations the dis-appearance of the molecule and the appearance of atomsare gradual The transition between two representationsis more easily comprehensible Cognition required inunderstanding the change of visual image can be kept atthe level of perceptual processing so that users can freetheir cognitive processing capacities for more compli-cated tasks (Robertson Card amp Mackinlay 1991)

Scale-based semantic representations can be seen asone member of the family of level of detail (LOD) tech-

niques which visualize the same structure with differentgeometric objects according to a particular interactionparameter but with a very different purpose Whilemost LOD techniques such as distance LOD primarilyconcern computation efficiency (Puppo amp Scopigno1997) scale-based semantic representation gives moreconsideration to the userrsquos interaction needs CommonLOD techniques render individual objects with differenttextures or geometries but scale-based semantic repre-sentation visualizes structures with different semanticabstractions LOD techniques can be integrated intoscale-based semantic representations to improve bothuser interaction and machine performance For examplein a city-planning project a city can have different struc-tures at the level of city district and street to informplanners semantically and distance LOD can be appliedin rendering individual objects in each structure to im-prove 3D computation

332 Dynamic View to Support Cross-ScaleInformation Sharing Design here addresses issuesconcerning how to help users at different scales to buildcommon understandings of their divergent views Onedesign choice is to use a dynamic view to narrow downthe view difference We used animation to show thetransition between two views Animation is generatedby interpolating the view positions view orientationsand view scales of two views For example if two usersrsquoviews at two different scales look like Figures 7(a) and7(b) an animation can be generated by adding interme-diate image frames between Figure 7(a) and 7(b) Theanimation can tell how one view can be transformed toanother and how objects in two views are related toeach other

333 Direct Avatar Manipulation to SupportCross-Scale Action To help users get involved inothersrsquo work so that they can take advantage of theirdifferent action capabilities we adopted a design choicethat allows direct manipulation on other usersrsquo avatarsWith such a tool planners in the above city-planningexample can easily cooperate on tasks that require largeaction domain and high action accuracy Of course thisdirect interference should be regulated The user who

Figure 8 Successive fading views in semantic representations

Zhang and Furnas 39

owns the avatar is always aware of such action and hascontrol over whether this action is allowed

The implication of such a direct manipulation of ava-tar body for user interface design deserves more atten-tion Under this design an avatar can ldquoaffordrdquo manipu-lation activities This is quite different from thetraditional role of an avatar merely as user embodimentin conventional CVEs The reason that no direct manip-ulation of an avatar is allowed in conventional CVEsmight be due to the influence of real life where the au-tonomy of human bodies is well respected To betterfacilitate peoplersquos work in virtual environments somereal-life constraints could be relaxed or even abandonedVirtual environments should not be treated as the repli-cation of the real world just as new technologies shouldnot be interpreted by old metaphors Direct manipula-tion of a userrsquos avatar by others is an effort to breakdown the metaphor of reality

4 System Implementation

A desktop mCVE prototype system consisting ofover 20000 lines of Java code was implemented Dis-cussions in this section focus on the choice of toolkits insoftware implementation general system architectureand the implementation of some multiscale tools

41 Choice of Toolkits

The design of virtual environments that supportcollaboration needs to consider issues in three aspects3D graphics rendering network communication man-agement and user interactions (Singhal amp Zyda 1999)It is not our interest to address issues related to low-level graphics rendering or network communicationsOur goal here is to support user interactions with multi-scale structures and with other users at different scalesTo achieve this interaction-oriented design goal weimplemented the mCVE prototype with two toolkitsJava 3D and Java Shared Data Toolkit (JSDT)

This hybrid approach allows a design focus on high-level interaction tools because Java 3D can handlegraphics rendering and JSDT can handle network com-

munications issues With the scene-graph managementof Java 3D we can construct virtual environments andstructures at the level of geometric objects rather thanat the level of vertices and polygons Java 3D allows theextension of new rendering behaviors which is not wellsupported by other CVE toolkits such as DIVE (Carls-son amp Hagsand 1993) and SPLINE (Waters et al1997) This flexibility is critical to the implementationof such tools as scale-based semantic representations anddynamic view transition Furthermore the easy integra-tion of other Java APIs such as Java Swing into Java3D can simplify user interface design JSDT is ideal forthe research because it provides flexible data deliveryservices in support of synchronous collaboration on theInternet JSDT supports basic abstraction of networksessions multicast message communications and con-currency control Thus the design can focus on opti-mizing the management of distributed 3D scenes andcross-scale communications One drawback of this hy-brid approach is the lack of those CVE services seen inSPLINE (Waters et al) Collaboration services have tobe built from scratch

42 System Layer and Architecture

The system layer of the prototype is shown in Fig-ure 9 The implementation focused on the two layersinside the dashed rectangle On top of Java 3D andJSDT are multiscale graphics services and collaboration

Figure 9 System layer of the mCVE prototype

40 PRESENCE VOLUME 14 NUMBER 1

services and above them are multiscale collaborationtools

Multiscale graphics services handle object-renderingissues and update user views based on their view posi-tions and interaction scales Collaboration services de-liver messages among distributed users and manage theconsistency of shared virtual environments Multiscalecollaboration services deal with such collaboration issuesas cross-scale information sharing and cross-scale avatarmanipulation These services lay the foundation for multi-scale tools

43 Implementation of Multiscale Tools

A set of multiscale collaboration tools were imple-mented and integrated into the prototype system Thissection introduces such tools as scaling control whichdistinguishes mCVEs from other CVEs scale-based se-mantic representations which visualize information in amultiscale way cross-scale dynamic view transitionwhich addresses the cross-scale information problemuniquely seen in multiscale collaboration and the man-agement of avatar position in direct avatar manipulationwhich illustrates cross-scale data coordination

431 Scene Graph of the Prototype Systemand Scaling Control The scene graph of the systemis shown in Figure 10 Each Java 3D scene graph has aVirtual Universe object as the root A Locale object at-taches scene graphs to the Virtual Universe objectBranchGroup (BG) nodes collect such objects as geom-etry behavior light sound and so forth A Transform-Group (TG) node defines the coordinates of objectsunder it

The system has two subscene graphs the object scenegraph which is the left child of the Locale object andthe view scene graph which is the right child The ob-ject scene graph collects all objects and behavior nodesThe view scene graph defines the viewing behaviors

To change a userrsquos interaction scale is to change thevalues of the userrsquos interaction parameters relative to thesize of the virtual world Thus scale change can be im-plemented in two ways (1) scaling the virtual world upand down and keeping all interaction parameters un-

touched or (2) changing all interaction parameters andkeeping the virtual world untouched Mathematicallythey are equivalent We adopted the first approach be-cause scaling the virtual world up and down can bedone by modifying just one parameter the scaling fac-tor Si of the top TG in the object scene graph This ismuch simpler than the second approach which involvesthe modification of complicated view parameters

432 Scale-Based Semantic RepresentationsIn the implementation of scale-based semantic represen-tations we adopted a design that uses a delegate objectto wrap all representations and that delivers appropriaterepresentations based on the value of the controllingvariable (Fox 1998) A scale-based semantic representa-tion node is extended from the Java 3D Switch objectEach node has a set of child representations and an or-dered array that defines the scale range for each repre-sentation (Figure 11)

433 Animating View Transition in Collabo-rative View Sharing The key issue in making a view-transition animation is to interpolate two views InmCVEs a view can be uniquely determined by its viewposition P its view orientation O and its view scale Sand be written as V(P O S)

Given two views V0 V(P0 O 0 S0) and V1 V(P1O 1 S1) to implement a view-transition animation is to

Figure 10 Scene graph of the prototype system

Zhang and Furnas 41

choose a path that links V0 and V1 and to set the trajec-tory of viewpoint movement view direction along thepath and observation scale along the path View-pathand view-orientation interpolation techniques are ma-ture (Parent 2001 Watt amp Policarpo 2001) so thisresearch focused only on the interpolation of view scale

A scale-interpolation function could be linear loga-rithmic or in other forms We chose a logarithmic func-tion because it gives users a constant relative rate ofchange in their views Interpolating the view scale loga-rithmically can give users a familiar experience

Assume the animation includes n frames from V0 toV1 The view scale of the ith frame (0 i n) in a log-arithmic interpolation function can be written as

Si Si S0 (1)

where

S S1

S01n

(2)

With corresponding P i and O i from interpolationfunctions for view position and view orientation theview of the ith frame Vi V(P i O i Si) can be ob-tained Figure 12 shows two views seen in Figure 7 asFrame 0 and Frame n and other interpolated viewframes (Due to space limitations we have shown onlyfour frames used for the animation) Showing theseframes successively with a reasonable frame rate can pro-duce an animation linking these two divergent views

434 Cross-Scale Data Consistency and Ava-tar Position in Direct Avatar Manipulation Al-though changing the scaling factor of the top TG in theobject scene graph simplifies the scaling operation such

an approach poses a challenge in cross-scale data consis-tency In CVEs all users usually need to see the sameobject appearing at the same spatial location In conven-tional CVEs this can be easily achieved because an ob-jectrsquos coordinates in different usersrsquo worlds are often thesame In mCVEs however when users are at differentscales their virtual worlds are rescaled according to thescaling factors of the top TG in their own object scenegraphs The same object will have different local coordi-nates in different usersrsquo worlds When the object ismoved by a user the objectrsquos local coordinates in thisuserrsquos world cannot be used to update the objectrsquos newposition in all other usersrsquo worlds

In the implementation we used normalized coordi-nates which are calculated as the ratio of the object lo-cal coordinates in the userrsquos world to the userrsquos scaleLet Pl(xl yl zl) and Sl denote the local coordinates andthe userrsquos scale respectively The normalized coordi-nates Pn can be written as

Pn PlSl (3)

By doing so the same object will always have the samePn This Pn is used to define the coordinates of the ob-ject in every userrsquos object scene graph Modifying thespatial location of an object in a userrsquos world leads to anew Pn All users can use this Pn to update the coordi-nates of the object in their object scene graphs There-fore Pn enforces data consistency across scale Becausethe scaling factor is on the top of the scene graph theobjectrsquos final location in a userrsquos world is Pl the productof Pn and this userrsquos scale Sl With different Sl users willbe able to see the world rendered at different scales

An avatarrsquos position can be managed similarly butwith a small difference An avatar object differs fromother ordinary objects in a way that in its host userrsquos

Figure 11 Scale-based semantic representation node

Figure 12 Frames of a dynamic view-transition animation

42 PRESENCE VOLUME 14 NUMBER 1

world the coordinates of the avatar are affiliated withthe view scene graph rather than the object scenegraph so that the avatar always moves with the view Inother usersrsquo worlds however the avatar object is underthe object scene graphs Figure 13(a) is the scene graphof the world of a user say A who interacts with the vir-tual environment at a location Pa and with a scale SaFigure 13(b) is the scene graph of the world of anotheruser say B User Arsquos avatar is in User Brsquos object graphBecause Pa is not governed by Sa the normalized coor-dinates of the avatar in User Brsquos object graph should bewritten as

PnPaSa (4)

The body size of User Arsquos avatar changes with thescale of User A so a scale parameter is also needed todefine the size of User Arsquos avatar in User Brsquos objectscene graph so that User B can be aware of User Arsquosscale This scale should also be normalized Let Sn de-note the normalized scale where Sn is simply the recip-rocal of Sa

Sn 1Sa (5)

This is because the avatar body changes inversely pro-portional to the scale factor Sa (shrinking the world isequivalent to growing the avatar body and growing theworld is equivalent to shrinking the avatar body)

The implementation of other cross-scale tools includ-ing direct avatar manipulation also uses Pn and Sn todefine the coordinates and size of an avatar entity Thisapproach guarantees the consistency of the avatar posi-tion and size in all usersrsquo views at different scales

5 Experimental Evaluation

We evaluated multiscale collaboration in subjecttests An experiment was designed to assess the benefitsusers can get from an mCVE in accomplishing tasks re-quiring cross-scale coordination

51 Experiment Design and Procedure

Subject tasks in the experiment involved searchingfor a ldquobombrdquo on a square ground plane (2000 2000m2) with a distinctly shaped building in each corner(square hexagon octagon and circle as seen fromabove) Each had a height of 12 m and a base of about6400 m2 On the ground behind each building was aunit cube (1 m3) containing a unique text name andsmiley face (about 05 05 m2) The bomb was insideone of these four cubes In the test subjects were placedin the middle of the square plane They had a defaulteye level of 168 m and a default moving step of 1 m inthe virtual environment The shape of the building thatthe bomb was nearby was known in advance and sub-jects needed to find that building locate the bomb boxand key in the name of the box to defuse the bomb

A 2 2 2 factorial design that has six treatmentstwo noncollaboration and four collaboration wasadopted (Table 1) For the two noncollaboration treat-ments VE is just a conventional 3D virtual environ-ment and M-VE is a VE enhanced by multiscale toolsspecifically it allows users to change their interactionscales eye level and speed Among four collaborationtreatments CVE a conventional collaborative virtualenvironment is the only one without multiscale toolsThe other three treatments all equipped with multiscale

Figure 13 Two scene graphs for the same avatar

Table 1 Six Treatments in a 2 2 2 Design

Noncollaboration Collaboration

Nonmultiscale VE CVEMultiscale M-VE NR

GUIDEMOVE

Zhang and Furnas 43

tools differ in the assignment of subject task roles (be-ing a giant or a normal person) and in the way subjectsaffect each otherrsquos work across scales In one treatmentthe roles of subjects are not predefined and subjects canchoose their own interaction scales as desired This envi-ronment is labeled as NR (No Role) The other twomCVEs both assign one subject to be a giant and theother a normal person so that subjects can change theirinteraction scales only within a limited range In onesuch environment the giant is permitted to move thepartner directly and this treatment is labeled as MOVEAnother condition allows the giant to guide the move-ment of the partner only verbally and is denoted asGUIDE

Recruited through email 24 students paired in 12groups participated in the experiment In noncollabora-tion treatments they worked on their own and in col-laboration treatments they communicated through anaudio channel The performances were measured bytask-completion time

52 Results

An ANOVA analysis of data from four treatments(VE M-VE CVE and MOVE) shows main effects ofboth collaboration (F170 1298 p 0001) andmultiscale (F170 7090 p 00001) Interaction isnot significant (F170 187 p 176) as seen in Fig-ure 14 Subjects performed best in MOVE where theycould take full advantage of multiscale and collabora-tion and they did worst in VE where there was no helpat all

Subjects used different strategies in the different con-ditions Without multiscale tools subjects had to goaround and count the number of sides of all four build-ings to find the target a very time-consuming processWith multiscale subjects could increase their sizes to seebuilding shapes from above and approach the bombquickly so the time required was be reduced signifi-cantly

The performance difference among the three mCVEsalso indicates the importance of different collaborationsupports In the NR treatment subjects had to assumetheir roles via an expensive negotiation process In the

GUIDE treatment the giant subject who could see thebuilding shape must coordinate with the partner aboutnavigation verbally through a costly grounding process(Clark amp Brennan 1991) In MOVE however the all-seeing giant could actually move the partner to the des-tination quickly and bypassed grounding processes

6 Discussion

In this paper we have introduced a novel interac-tive environment multiscale collaborative virtual envi-ronment which gives users the freedom to control theirinteraction scales in collaboration Multiscale collabora-tion can help better allocate scarce scale resources andcombine different expertise and action capabilities ofindividual users in their work with large structures Wehave also discussed the implications of multiscale tech-nology for user interaction and the design and develop-ment of a Java-based mCVE system Our subject testshave shown that multiscale collaboration can facilitatepeoplersquos work that requires cross-scale information andcollaboration

Research reported in this paper has largely focused ongeneral interaction issues in mCVEs Designs have beenlimited to generic tools to support multiscale collabora-tion and experimental study has focused on generaltasks We expect that multiscale collaboration tools canbe used to support peoplersquos specific work in the realworld To achieve this goal the design of multiscale

Figure 14 Time (in seconds) for the six treatments

44 PRESENCE VOLUME 14 NUMBER 1

tools should support domain-specific tasks One of ourongoing research projects focuses on studying how multi-scale collaboration can support research in materials sci-ence and engineering We are investigating the needs ofmaterials scientists for multiscale collaboration and willdesign and implement tools that support these needsThe effectiveness and usability of these tools will also beevaluated when they are ready

At the same time we are interested in behavior issuesin mCVEs Some important research questions concern-ing user behavior in mCVEs have not been studiedthoroughly It is still unclear when and how users canbenefit from collaborative object manipulation acrossscale whether the designed dynamic view-transitiontechnique can indeed improve cross-scale informationsharing what mechanism is needed to help users bettermanage cross-scale collaboration overheads and soforth In addition it has been found that multiscale alsobrings some new challenges in social interactions(Zhang amp Furnas 2002) and efforts are needed tostudy what can be done to address these interaction is-sues Research in this direction will deepen our under-standing of mCVEs and help to design better tools tosupport multiscale collaboration

Acknowledgments

This research was funded in part by Microsoft Research

References

Ahl V amp Allen T (1996) Hierarchy theory A vision vocab-ulary and epistemology New York Columbia UniversityPress

Alexander C Ishikawa S amp Silverstein M (1977) A pat-tern language Oxford UK Oxford University Press

Bederson B amp Hollan J (1994) Pad A zoominggraphical interface for exploring alternate interface physicsProceedings of ACM UIST rsquo94 17ndash26

Benford S Bowers J Fahlen L Greenhalgh C amp Snow-don D (1995) User embodiment in collaborative virtualenvironments Proceedings of ACM CHI rsquo95 242ndash249

Benford S Snowdon D Colebourne A OrsquoBrien J ampRodden T (1997) Informing the design of collaborativevirtual environments Proceedings of ACM GROUP rsquo97 71ndash80

Carlsson C amp Hagsand O (1993) DIVEmdashA platform formulti-user virtual environments Computer and Graphics17(6) 663ndash669

Clark H amp Brennan S (1991) Grounding in communica-tion In L B Resnick J Levine amp S D Behrend (Eds)Perspectives on socially shared cognition (pp 127ndash149)Washington DC American Psychological Association

Dix A J (1994) Computer-supported cooperative workmdashAframework In D Rosenburg amp C Hutchison (Eds) De-sign Issues in CSCW (pp 23ndash37) Berlin Springer-Verlag

EPFL Geneva IIS TNO Lancaster UCL et al (1997)Guidelines for building CVE applications COVEN ProjectRetrieved November 21 2002 from httpwwwcsuclacukstaffmslatervrCovenPublicDeliverablesdel26PDF

Fox D (1998) Tabula rasa A multi-scale user interface sys-tem Unpublished doctoral dissertation New York Univer-sity

Furnas G (1986) Generalized fisheye views Proceedings ofACM CHI rsquo86 16ndash23

Gaver W Sellen A Heath C amp Luff P (1993) One isnot enough Multiple views in a media space Proceedings ofACM INTERCHI rsquo93 335ndash341

Hollan J amp Stornetta S (1992) Beyond being there Pro-ceedings of ACM CHI rsquo92 119ndash125

Koike H amp Yoshihara H (1993) Fractal approaches forvisualizing huge hierarchies Proceedings of IEEE VL rsquo9355ndash60

Lamping J Rao R amp Pirolli P (1995) A focus contexttechnique based on hyperbolic geometry for visualizinglarge hierarchies Proceedings of ACM CHI rsquo95 401ndash408

Leigh J Johnson A amp DeFanti T (1996) CALVIN Animmersimedia design environment utilizing heterogeneousperspectives Proceedings of IEEE International Conferenceon Multimedia Computing and Systems 20ndash23

Lieberman H (1994) Powers of ten thousand Navigating inlarge information spaces Proceedings of ACM UIST rsquo9415ndash16

Mackinlay J Card S amp Robertson G (1990) Rapid con-trolled movement through a virtual 3D workspace Proceed-ings of ACM SIGGRAPH rsquo90 171ndash176

Mine M Brooks F amp Sequin C (1997) Moving objects

Zhang and Furnas 45

in space Exploiting proprioception in virtual-environmentinteraction Proceedings of ACM SIGGRAPH rsquo97 19ndash26

Morrison P amp Morrison P (1994) Powers of ten About therelative size of things in the universe and the effect of addinganother zero San Francisco W H Freeman

Parent R (2001) Computer animation Algorithms and tech-niques San Francisco Morgan-Kaufmann

Pattee H (1973) Hierarchy theory The challenge of complexsystems New York George Braziller

Perlin K amp Fox D (1993) Pad An alternative approach tothe computer interface Proceedings of ACM SIGGRAPHrsquo93 57ndash64

Pierce J Forsberg A Conway M Hong S Zeleznik Ramp Mine M (1997) Image plane interaction techniques in3D immersive environments Proceedings of the 1997 Sympo-sium on Interactive 3D Graphics 39ndash43

Poupyrev I Billinghurst M Weghorst S amp Ichikawa T(1996) The go-go interaction technique Non-linear map-ping for direct manipulation in VR Proceedings of ACMUIST rsquo96 79ndash80

Puppo E amp Scopigno R (1997) Simplification LOD multi-resolution Principles and applications Proceedings ofEUROGRAPHICS 16 (3)

Raab A amp Ruger M (1996) 3D-Zoom Interactive visual-ization of structures and relations in complex graphics InB Girod H Nieman amp H Seidel (Eds) 3D image analy-sis and synthesis (pp 125ndash132) Sankt Augustin GermanyInfix-Verlag

Robbins M (2001) Modeling and simulation of the failureand toughness of polymersubstrate bonds The second An-nual Multiscale ModelingmdashBridging the Gap Between Atom-istic and Meso Scales

Robertson G Card S amp Mackinlay J (1991) Informationvisualization using 3D interactive animation Proceedings ofACM CHI rsquo91 461ndash462

Robertson G amp Mackinlay J (1993) The document lensProceedings of ACM UIST rsquo93 101ndash110

Robertson G Mackinlay J amp Card S (1991) Cone treesAnimated 3D visualizations of hierarchical informationCommunications of the ACM 34 (2) p 89ndash194

Sarkar M amp Brown M H (1992) Graphical fisheye viewsof graphs Proceedings of ACM CHI rsquo92 83ndash91

Singhal S amp Zyda M (1999) Networked virtual environ-ments Design and implementation New York ACM Press

Snowdon D amp Jaa-Aro K (1997) A subjective virtual envi-ronment for collaborative information visualization VirtualReality Universe rsquo97 Retrieved May 23 2003 fromhttpwwwcyberedgecomvru_paperssnowdenhtm

Stappers P Gaver W amp Overbeeke C (2000) Beyond thelimits of real-time realism 2000 Chapter to appear in L JHettinger amp M W Haas (Eds) Psychological issues in thedesign and use of virtual and adaptive environment Mah-wah NJ Erlbaum

Stefik M Bobrow D Foster G Lanning S amp Tatar D(1987) WYSIWIS revised Early experiences with multiuserinterfaces ACM Transactions on Information Systems(TOIS) 5(2) 147ndash167

Stoakley R Conway M amp Pausch R (1995) Virtual real-ity on a WIM Interactive worlds in miniature Proceedingsof ACM CHI rsquo95 265ndash272

Sutherland I (1965) The ultimate display Proceedings ofIFIP Congress 2

Tromp J G (1999) Usability design for CVEs COVENProject Retrieved November 21 2002 from httpcovenlancsacuk4deliverablesdel29pdf

Waters R Anderson D Barrus J Brogan D Casey MMcKeown S et al (1997) Diamond park and SPLINESocial virtual reality with 3D animation spoken interactionand runtime extendibility Presence Teleoperators and Vir-tual Environments 6(4) 461ndash481

Watt A amp Policarpo F (2001) 3D games Animation andadvanced real-time rendering New York Addison-Wesley

Zhang X amp Furnas G (2002) Social interactions in multi-scale CVEs Proceedings of ACM CVE 2002 31ndash38

46 PRESENCE VOLUME 14 NUMBER 1

owns the avatar is always aware of such action and hascontrol over whether this action is allowed

The implication of such a direct manipulation of ava-tar body for user interface design deserves more atten-tion Under this design an avatar can ldquoaffordrdquo manipu-lation activities This is quite different from thetraditional role of an avatar merely as user embodimentin conventional CVEs The reason that no direct manip-ulation of an avatar is allowed in conventional CVEsmight be due to the influence of real life where the au-tonomy of human bodies is well respected To betterfacilitate peoplersquos work in virtual environments somereal-life constraints could be relaxed or even abandonedVirtual environments should not be treated as the repli-cation of the real world just as new technologies shouldnot be interpreted by old metaphors Direct manipula-tion of a userrsquos avatar by others is an effort to breakdown the metaphor of reality

4 System Implementation

A desktop mCVE prototype system consisting ofover 20000 lines of Java code was implemented Dis-cussions in this section focus on the choice of toolkits insoftware implementation general system architectureand the implementation of some multiscale tools

41 Choice of Toolkits

The design of virtual environments that supportcollaboration needs to consider issues in three aspects3D graphics rendering network communication man-agement and user interactions (Singhal amp Zyda 1999)It is not our interest to address issues related to low-level graphics rendering or network communicationsOur goal here is to support user interactions with multi-scale structures and with other users at different scalesTo achieve this interaction-oriented design goal weimplemented the mCVE prototype with two toolkitsJava 3D and Java Shared Data Toolkit (JSDT)

This hybrid approach allows a design focus on high-level interaction tools because Java 3D can handlegraphics rendering and JSDT can handle network com-

munications issues With the scene-graph managementof Java 3D we can construct virtual environments andstructures at the level of geometric objects rather thanat the level of vertices and polygons Java 3D allows theextension of new rendering behaviors which is not wellsupported by other CVE toolkits such as DIVE (Carls-son amp Hagsand 1993) and SPLINE (Waters et al1997) This flexibility is critical to the implementationof such tools as scale-based semantic representations anddynamic view transition Furthermore the easy integra-tion of other Java APIs such as Java Swing into Java3D can simplify user interface design JSDT is ideal forthe research because it provides flexible data deliveryservices in support of synchronous collaboration on theInternet JSDT supports basic abstraction of networksessions multicast message communications and con-currency control Thus the design can focus on opti-mizing the management of distributed 3D scenes andcross-scale communications One drawback of this hy-brid approach is the lack of those CVE services seen inSPLINE (Waters et al) Collaboration services have tobe built from scratch

42 System Layer and Architecture

The system layer of the prototype is shown in Fig-ure 9 The implementation focused on the two layersinside the dashed rectangle On top of Java 3D andJSDT are multiscale graphics services and collaboration

Figure 9 System layer of the mCVE prototype

40 PRESENCE VOLUME 14 NUMBER 1

services and above them are multiscale collaborationtools

Multiscale graphics services handle object-renderingissues and update user views based on their view posi-tions and interaction scales Collaboration services de-liver messages among distributed users and manage theconsistency of shared virtual environments Multiscalecollaboration services deal with such collaboration issuesas cross-scale information sharing and cross-scale avatarmanipulation These services lay the foundation for multi-scale tools

43 Implementation of Multiscale Tools

A set of multiscale collaboration tools were imple-mented and integrated into the prototype system Thissection introduces such tools as scaling control whichdistinguishes mCVEs from other CVEs scale-based se-mantic representations which visualize information in amultiscale way cross-scale dynamic view transitionwhich addresses the cross-scale information problemuniquely seen in multiscale collaboration and the man-agement of avatar position in direct avatar manipulationwhich illustrates cross-scale data coordination

431 Scene Graph of the Prototype Systemand Scaling Control The scene graph of the systemis shown in Figure 10 Each Java 3D scene graph has aVirtual Universe object as the root A Locale object at-taches scene graphs to the Virtual Universe objectBranchGroup (BG) nodes collect such objects as geom-etry behavior light sound and so forth A Transform-Group (TG) node defines the coordinates of objectsunder it

The system has two subscene graphs the object scenegraph which is the left child of the Locale object andthe view scene graph which is the right child The ob-ject scene graph collects all objects and behavior nodesThe view scene graph defines the viewing behaviors

To change a userrsquos interaction scale is to change thevalues of the userrsquos interaction parameters relative to thesize of the virtual world Thus scale change can be im-plemented in two ways (1) scaling the virtual world upand down and keeping all interaction parameters un-

touched or (2) changing all interaction parameters andkeeping the virtual world untouched Mathematicallythey are equivalent We adopted the first approach be-cause scaling the virtual world up and down can bedone by modifying just one parameter the scaling fac-tor Si of the top TG in the object scene graph This ismuch simpler than the second approach which involvesthe modification of complicated view parameters

432 Scale-Based Semantic RepresentationsIn the implementation of scale-based semantic represen-tations we adopted a design that uses a delegate objectto wrap all representations and that delivers appropriaterepresentations based on the value of the controllingvariable (Fox 1998) A scale-based semantic representa-tion node is extended from the Java 3D Switch objectEach node has a set of child representations and an or-dered array that defines the scale range for each repre-sentation (Figure 11)

433 Animating View Transition in Collabo-rative View Sharing The key issue in making a view-transition animation is to interpolate two views InmCVEs a view can be uniquely determined by its viewposition P its view orientation O and its view scale Sand be written as V(P O S)

Given two views V0 V(P0 O 0 S0) and V1 V(P1O 1 S1) to implement a view-transition animation is to

Figure 10 Scene graph of the prototype system

Zhang and Furnas 41

choose a path that links V0 and V1 and to set the trajec-tory of viewpoint movement view direction along thepath and observation scale along the path View-pathand view-orientation interpolation techniques are ma-ture (Parent 2001 Watt amp Policarpo 2001) so thisresearch focused only on the interpolation of view scale

A scale-interpolation function could be linear loga-rithmic or in other forms We chose a logarithmic func-tion because it gives users a constant relative rate ofchange in their views Interpolating the view scale loga-rithmically can give users a familiar experience

Assume the animation includes n frames from V0 toV1 The view scale of the ith frame (0 i n) in a log-arithmic interpolation function can be written as

Si Si S0 (1)

where

S S1

S01n

(2)

With corresponding P i and O i from interpolationfunctions for view position and view orientation theview of the ith frame Vi V(P i O i Si) can be ob-tained Figure 12 shows two views seen in Figure 7 asFrame 0 and Frame n and other interpolated viewframes (Due to space limitations we have shown onlyfour frames used for the animation) Showing theseframes successively with a reasonable frame rate can pro-duce an animation linking these two divergent views

434 Cross-Scale Data Consistency and Ava-tar Position in Direct Avatar Manipulation Al-though changing the scaling factor of the top TG in theobject scene graph simplifies the scaling operation such

an approach poses a challenge in cross-scale data consis-tency In CVEs all users usually need to see the sameobject appearing at the same spatial location In conven-tional CVEs this can be easily achieved because an ob-jectrsquos coordinates in different usersrsquo worlds are often thesame In mCVEs however when users are at differentscales their virtual worlds are rescaled according to thescaling factors of the top TG in their own object scenegraphs The same object will have different local coordi-nates in different usersrsquo worlds When the object ismoved by a user the objectrsquos local coordinates in thisuserrsquos world cannot be used to update the objectrsquos newposition in all other usersrsquo worlds

In the implementation we used normalized coordi-nates which are calculated as the ratio of the object lo-cal coordinates in the userrsquos world to the userrsquos scaleLet Pl(xl yl zl) and Sl denote the local coordinates andthe userrsquos scale respectively The normalized coordi-nates Pn can be written as

Pn PlSl (3)

By doing so the same object will always have the samePn This Pn is used to define the coordinates of the ob-ject in every userrsquos object scene graph Modifying thespatial location of an object in a userrsquos world leads to anew Pn All users can use this Pn to update the coordi-nates of the object in their object scene graphs There-fore Pn enforces data consistency across scale Becausethe scaling factor is on the top of the scene graph theobjectrsquos final location in a userrsquos world is Pl the productof Pn and this userrsquos scale Sl With different Sl users willbe able to see the world rendered at different scales

An avatarrsquos position can be managed similarly butwith a small difference An avatar object differs fromother ordinary objects in a way that in its host userrsquos

Figure 11 Scale-based semantic representation node

Figure 12 Frames of a dynamic view-transition animation

42 PRESENCE VOLUME 14 NUMBER 1

world the coordinates of the avatar are affiliated withthe view scene graph rather than the object scenegraph so that the avatar always moves with the view Inother usersrsquo worlds however the avatar object is underthe object scene graphs Figure 13(a) is the scene graphof the world of a user say A who interacts with the vir-tual environment at a location Pa and with a scale SaFigure 13(b) is the scene graph of the world of anotheruser say B User Arsquos avatar is in User Brsquos object graphBecause Pa is not governed by Sa the normalized coor-dinates of the avatar in User Brsquos object graph should bewritten as

PnPaSa (4)

The body size of User Arsquos avatar changes with thescale of User A so a scale parameter is also needed todefine the size of User Arsquos avatar in User Brsquos objectscene graph so that User B can be aware of User Arsquosscale This scale should also be normalized Let Sn de-note the normalized scale where Sn is simply the recip-rocal of Sa

Sn 1Sa (5)

This is because the avatar body changes inversely pro-portional to the scale factor Sa (shrinking the world isequivalent to growing the avatar body and growing theworld is equivalent to shrinking the avatar body)

The implementation of other cross-scale tools includ-ing direct avatar manipulation also uses Pn and Sn todefine the coordinates and size of an avatar entity Thisapproach guarantees the consistency of the avatar posi-tion and size in all usersrsquo views at different scales

5 Experimental Evaluation

We evaluated multiscale collaboration in subjecttests An experiment was designed to assess the benefitsusers can get from an mCVE in accomplishing tasks re-quiring cross-scale coordination

51 Experiment Design and Procedure

Subject tasks in the experiment involved searchingfor a ldquobombrdquo on a square ground plane (2000 2000m2) with a distinctly shaped building in each corner(square hexagon octagon and circle as seen fromabove) Each had a height of 12 m and a base of about6400 m2 On the ground behind each building was aunit cube (1 m3) containing a unique text name andsmiley face (about 05 05 m2) The bomb was insideone of these four cubes In the test subjects were placedin the middle of the square plane They had a defaulteye level of 168 m and a default moving step of 1 m inthe virtual environment The shape of the building thatthe bomb was nearby was known in advance and sub-jects needed to find that building locate the bomb boxand key in the name of the box to defuse the bomb

A 2 2 2 factorial design that has six treatmentstwo noncollaboration and four collaboration wasadopted (Table 1) For the two noncollaboration treat-ments VE is just a conventional 3D virtual environ-ment and M-VE is a VE enhanced by multiscale toolsspecifically it allows users to change their interactionscales eye level and speed Among four collaborationtreatments CVE a conventional collaborative virtualenvironment is the only one without multiscale toolsThe other three treatments all equipped with multiscale

Figure 13 Two scene graphs for the same avatar

Table 1 Six Treatments in a 2 2 2 Design

Noncollaboration Collaboration

Nonmultiscale VE CVEMultiscale M-VE NR

GUIDEMOVE

Zhang and Furnas 43

tools differ in the assignment of subject task roles (be-ing a giant or a normal person) and in the way subjectsaffect each otherrsquos work across scales In one treatmentthe roles of subjects are not predefined and subjects canchoose their own interaction scales as desired This envi-ronment is labeled as NR (No Role) The other twomCVEs both assign one subject to be a giant and theother a normal person so that subjects can change theirinteraction scales only within a limited range In onesuch environment the giant is permitted to move thepartner directly and this treatment is labeled as MOVEAnother condition allows the giant to guide the move-ment of the partner only verbally and is denoted asGUIDE

Recruited through email 24 students paired in 12groups participated in the experiment In noncollabora-tion treatments they worked on their own and in col-laboration treatments they communicated through anaudio channel The performances were measured bytask-completion time

52 Results

An ANOVA analysis of data from four treatments(VE M-VE CVE and MOVE) shows main effects ofboth collaboration (F170 1298 p 0001) andmultiscale (F170 7090 p 00001) Interaction isnot significant (F170 187 p 176) as seen in Fig-ure 14 Subjects performed best in MOVE where theycould take full advantage of multiscale and collabora-tion and they did worst in VE where there was no helpat all

Subjects used different strategies in the different con-ditions Without multiscale tools subjects had to goaround and count the number of sides of all four build-ings to find the target a very time-consuming processWith multiscale subjects could increase their sizes to seebuilding shapes from above and approach the bombquickly so the time required was be reduced signifi-cantly

The performance difference among the three mCVEsalso indicates the importance of different collaborationsupports In the NR treatment subjects had to assumetheir roles via an expensive negotiation process In the

GUIDE treatment the giant subject who could see thebuilding shape must coordinate with the partner aboutnavigation verbally through a costly grounding process(Clark amp Brennan 1991) In MOVE however the all-seeing giant could actually move the partner to the des-tination quickly and bypassed grounding processes

6 Discussion

In this paper we have introduced a novel interac-tive environment multiscale collaborative virtual envi-ronment which gives users the freedom to control theirinteraction scales in collaboration Multiscale collabora-tion can help better allocate scarce scale resources andcombine different expertise and action capabilities ofindividual users in their work with large structures Wehave also discussed the implications of multiscale tech-nology for user interaction and the design and develop-ment of a Java-based mCVE system Our subject testshave shown that multiscale collaboration can facilitatepeoplersquos work that requires cross-scale information andcollaboration

Research reported in this paper has largely focused ongeneral interaction issues in mCVEs Designs have beenlimited to generic tools to support multiscale collabora-tion and experimental study has focused on generaltasks We expect that multiscale collaboration tools canbe used to support peoplersquos specific work in the realworld To achieve this goal the design of multiscale

Figure 14 Time (in seconds) for the six treatments

44 PRESENCE VOLUME 14 NUMBER 1

tools should support domain-specific tasks One of ourongoing research projects focuses on studying how multi-scale collaboration can support research in materials sci-ence and engineering We are investigating the needs ofmaterials scientists for multiscale collaboration and willdesign and implement tools that support these needsThe effectiveness and usability of these tools will also beevaluated when they are ready

At the same time we are interested in behavior issuesin mCVEs Some important research questions concern-ing user behavior in mCVEs have not been studiedthoroughly It is still unclear when and how users canbenefit from collaborative object manipulation acrossscale whether the designed dynamic view-transitiontechnique can indeed improve cross-scale informationsharing what mechanism is needed to help users bettermanage cross-scale collaboration overheads and soforth In addition it has been found that multiscale alsobrings some new challenges in social interactions(Zhang amp Furnas 2002) and efforts are needed tostudy what can be done to address these interaction is-sues Research in this direction will deepen our under-standing of mCVEs and help to design better tools tosupport multiscale collaboration

Acknowledgments

This research was funded in part by Microsoft Research

References

Ahl V amp Allen T (1996) Hierarchy theory A vision vocab-ulary and epistemology New York Columbia UniversityPress

Alexander C Ishikawa S amp Silverstein M (1977) A pat-tern language Oxford UK Oxford University Press

Bederson B amp Hollan J (1994) Pad A zoominggraphical interface for exploring alternate interface physicsProceedings of ACM UIST rsquo94 17ndash26

Benford S Bowers J Fahlen L Greenhalgh C amp Snow-don D (1995) User embodiment in collaborative virtualenvironments Proceedings of ACM CHI rsquo95 242ndash249

Benford S Snowdon D Colebourne A OrsquoBrien J ampRodden T (1997) Informing the design of collaborativevirtual environments Proceedings of ACM GROUP rsquo97 71ndash80

Carlsson C amp Hagsand O (1993) DIVEmdashA platform formulti-user virtual environments Computer and Graphics17(6) 663ndash669

Clark H amp Brennan S (1991) Grounding in communica-tion In L B Resnick J Levine amp S D Behrend (Eds)Perspectives on socially shared cognition (pp 127ndash149)Washington DC American Psychological Association

Dix A J (1994) Computer-supported cooperative workmdashAframework In D Rosenburg amp C Hutchison (Eds) De-sign Issues in CSCW (pp 23ndash37) Berlin Springer-Verlag

EPFL Geneva IIS TNO Lancaster UCL et al (1997)Guidelines for building CVE applications COVEN ProjectRetrieved November 21 2002 from httpwwwcsuclacukstaffmslatervrCovenPublicDeliverablesdel26PDF

Fox D (1998) Tabula rasa A multi-scale user interface sys-tem Unpublished doctoral dissertation New York Univer-sity

Furnas G (1986) Generalized fisheye views Proceedings ofACM CHI rsquo86 16ndash23

Gaver W Sellen A Heath C amp Luff P (1993) One isnot enough Multiple views in a media space Proceedings ofACM INTERCHI rsquo93 335ndash341

Hollan J amp Stornetta S (1992) Beyond being there Pro-ceedings of ACM CHI rsquo92 119ndash125

Koike H amp Yoshihara H (1993) Fractal approaches forvisualizing huge hierarchies Proceedings of IEEE VL rsquo9355ndash60

Lamping J Rao R amp Pirolli P (1995) A focus contexttechnique based on hyperbolic geometry for visualizinglarge hierarchies Proceedings of ACM CHI rsquo95 401ndash408

Leigh J Johnson A amp DeFanti T (1996) CALVIN Animmersimedia design environment utilizing heterogeneousperspectives Proceedings of IEEE International Conferenceon Multimedia Computing and Systems 20ndash23

Lieberman H (1994) Powers of ten thousand Navigating inlarge information spaces Proceedings of ACM UIST rsquo9415ndash16

Mackinlay J Card S amp Robertson G (1990) Rapid con-trolled movement through a virtual 3D workspace Proceed-ings of ACM SIGGRAPH rsquo90 171ndash176

Mine M Brooks F amp Sequin C (1997) Moving objects

Zhang and Furnas 45

in space Exploiting proprioception in virtual-environmentinteraction Proceedings of ACM SIGGRAPH rsquo97 19ndash26

Morrison P amp Morrison P (1994) Powers of ten About therelative size of things in the universe and the effect of addinganother zero San Francisco W H Freeman

Parent R (2001) Computer animation Algorithms and tech-niques San Francisco Morgan-Kaufmann

Pattee H (1973) Hierarchy theory The challenge of complexsystems New York George Braziller

Perlin K amp Fox D (1993) Pad An alternative approach tothe computer interface Proceedings of ACM SIGGRAPHrsquo93 57ndash64

Pierce J Forsberg A Conway M Hong S Zeleznik Ramp Mine M (1997) Image plane interaction techniques in3D immersive environments Proceedings of the 1997 Sympo-sium on Interactive 3D Graphics 39ndash43

Poupyrev I Billinghurst M Weghorst S amp Ichikawa T(1996) The go-go interaction technique Non-linear map-ping for direct manipulation in VR Proceedings of ACMUIST rsquo96 79ndash80

Puppo E amp Scopigno R (1997) Simplification LOD multi-resolution Principles and applications Proceedings ofEUROGRAPHICS 16 (3)

Raab A amp Ruger M (1996) 3D-Zoom Interactive visual-ization of structures and relations in complex graphics InB Girod H Nieman amp H Seidel (Eds) 3D image analy-sis and synthesis (pp 125ndash132) Sankt Augustin GermanyInfix-Verlag

Robbins M (2001) Modeling and simulation of the failureand toughness of polymersubstrate bonds The second An-nual Multiscale ModelingmdashBridging the Gap Between Atom-istic and Meso Scales

Robertson G Card S amp Mackinlay J (1991) Informationvisualization using 3D interactive animation Proceedings ofACM CHI rsquo91 461ndash462

Robertson G amp Mackinlay J (1993) The document lensProceedings of ACM UIST rsquo93 101ndash110

Robertson G Mackinlay J amp Card S (1991) Cone treesAnimated 3D visualizations of hierarchical informationCommunications of the ACM 34 (2) p 89ndash194

Sarkar M amp Brown M H (1992) Graphical fisheye viewsof graphs Proceedings of ACM CHI rsquo92 83ndash91

Singhal S amp Zyda M (1999) Networked virtual environ-ments Design and implementation New York ACM Press

Snowdon D amp Jaa-Aro K (1997) A subjective virtual envi-ronment for collaborative information visualization VirtualReality Universe rsquo97 Retrieved May 23 2003 fromhttpwwwcyberedgecomvru_paperssnowdenhtm

Stappers P Gaver W amp Overbeeke C (2000) Beyond thelimits of real-time realism 2000 Chapter to appear in L JHettinger amp M W Haas (Eds) Psychological issues in thedesign and use of virtual and adaptive environment Mah-wah NJ Erlbaum

Stefik M Bobrow D Foster G Lanning S amp Tatar D(1987) WYSIWIS revised Early experiences with multiuserinterfaces ACM Transactions on Information Systems(TOIS) 5(2) 147ndash167

Stoakley R Conway M amp Pausch R (1995) Virtual real-ity on a WIM Interactive worlds in miniature Proceedingsof ACM CHI rsquo95 265ndash272

Sutherland I (1965) The ultimate display Proceedings ofIFIP Congress 2

Tromp J G (1999) Usability design for CVEs COVENProject Retrieved November 21 2002 from httpcovenlancsacuk4deliverablesdel29pdf

Waters R Anderson D Barrus J Brogan D Casey MMcKeown S et al (1997) Diamond park and SPLINESocial virtual reality with 3D animation spoken interactionand runtime extendibility Presence Teleoperators and Vir-tual Environments 6(4) 461ndash481

Watt A amp Policarpo F (2001) 3D games Animation andadvanced real-time rendering New York Addison-Wesley

Zhang X amp Furnas G (2002) Social interactions in multi-scale CVEs Proceedings of ACM CVE 2002 31ndash38

46 PRESENCE VOLUME 14 NUMBER 1

services and above them are multiscale collaborationtools

Multiscale graphics services handle object-renderingissues and update user views based on their view posi-tions and interaction scales Collaboration services de-liver messages among distributed users and manage theconsistency of shared virtual environments Multiscalecollaboration services deal with such collaboration issuesas cross-scale information sharing and cross-scale avatarmanipulation These services lay the foundation for multi-scale tools

43 Implementation of Multiscale Tools

A set of multiscale collaboration tools were imple-mented and integrated into the prototype system Thissection introduces such tools as scaling control whichdistinguishes mCVEs from other CVEs scale-based se-mantic representations which visualize information in amultiscale way cross-scale dynamic view transitionwhich addresses the cross-scale information problemuniquely seen in multiscale collaboration and the man-agement of avatar position in direct avatar manipulationwhich illustrates cross-scale data coordination

431 Scene Graph of the Prototype Systemand Scaling Control The scene graph of the systemis shown in Figure 10 Each Java 3D scene graph has aVirtual Universe object as the root A Locale object at-taches scene graphs to the Virtual Universe objectBranchGroup (BG) nodes collect such objects as geom-etry behavior light sound and so forth A Transform-Group (TG) node defines the coordinates of objectsunder it

The system has two subscene graphs the object scenegraph which is the left child of the Locale object andthe view scene graph which is the right child The ob-ject scene graph collects all objects and behavior nodesThe view scene graph defines the viewing behaviors

To change a userrsquos interaction scale is to change thevalues of the userrsquos interaction parameters relative to thesize of the virtual world Thus scale change can be im-plemented in two ways (1) scaling the virtual world upand down and keeping all interaction parameters un-

touched or (2) changing all interaction parameters andkeeping the virtual world untouched Mathematicallythey are equivalent We adopted the first approach be-cause scaling the virtual world up and down can bedone by modifying just one parameter the scaling fac-tor Si of the top TG in the object scene graph This ismuch simpler than the second approach which involvesthe modification of complicated view parameters

432 Scale-Based Semantic RepresentationsIn the implementation of scale-based semantic represen-tations we adopted a design that uses a delegate objectto wrap all representations and that delivers appropriaterepresentations based on the value of the controllingvariable (Fox 1998) A scale-based semantic representa-tion node is extended from the Java 3D Switch objectEach node has a set of child representations and an or-dered array that defines the scale range for each repre-sentation (Figure 11)

433 Animating View Transition in Collabo-rative View Sharing The key issue in making a view-transition animation is to interpolate two views InmCVEs a view can be uniquely determined by its viewposition P its view orientation O and its view scale Sand be written as V(P O S)

Given two views V0 V(P0 O 0 S0) and V1 V(P1O 1 S1) to implement a view-transition animation is to

Figure 10 Scene graph of the prototype system

Zhang and Furnas 41

choose a path that links V0 and V1 and to set the trajec-tory of viewpoint movement view direction along thepath and observation scale along the path View-pathand view-orientation interpolation techniques are ma-ture (Parent 2001 Watt amp Policarpo 2001) so thisresearch focused only on the interpolation of view scale

A scale-interpolation function could be linear loga-rithmic or in other forms We chose a logarithmic func-tion because it gives users a constant relative rate ofchange in their views Interpolating the view scale loga-rithmically can give users a familiar experience

Assume the animation includes n frames from V0 toV1 The view scale of the ith frame (0 i n) in a log-arithmic interpolation function can be written as

Si Si S0 (1)

where

S S1

S01n

(2)

With corresponding P i and O i from interpolationfunctions for view position and view orientation theview of the ith frame Vi V(P i O i Si) can be ob-tained Figure 12 shows two views seen in Figure 7 asFrame 0 and Frame n and other interpolated viewframes (Due to space limitations we have shown onlyfour frames used for the animation) Showing theseframes successively with a reasonable frame rate can pro-duce an animation linking these two divergent views

434 Cross-Scale Data Consistency and Ava-tar Position in Direct Avatar Manipulation Al-though changing the scaling factor of the top TG in theobject scene graph simplifies the scaling operation such

an approach poses a challenge in cross-scale data consis-tency In CVEs all users usually need to see the sameobject appearing at the same spatial location In conven-tional CVEs this can be easily achieved because an ob-jectrsquos coordinates in different usersrsquo worlds are often thesame In mCVEs however when users are at differentscales their virtual worlds are rescaled according to thescaling factors of the top TG in their own object scenegraphs The same object will have different local coordi-nates in different usersrsquo worlds When the object ismoved by a user the objectrsquos local coordinates in thisuserrsquos world cannot be used to update the objectrsquos newposition in all other usersrsquo worlds

In the implementation we used normalized coordi-nates which are calculated as the ratio of the object lo-cal coordinates in the userrsquos world to the userrsquos scaleLet Pl(xl yl zl) and Sl denote the local coordinates andthe userrsquos scale respectively The normalized coordi-nates Pn can be written as

Pn PlSl (3)

By doing so the same object will always have the samePn This Pn is used to define the coordinates of the ob-ject in every userrsquos object scene graph Modifying thespatial location of an object in a userrsquos world leads to anew Pn All users can use this Pn to update the coordi-nates of the object in their object scene graphs There-fore Pn enforces data consistency across scale Becausethe scaling factor is on the top of the scene graph theobjectrsquos final location in a userrsquos world is Pl the productof Pn and this userrsquos scale Sl With different Sl users willbe able to see the world rendered at different scales

An avatarrsquos position can be managed similarly butwith a small difference An avatar object differs fromother ordinary objects in a way that in its host userrsquos

Figure 11 Scale-based semantic representation node

Figure 12 Frames of a dynamic view-transition animation

42 PRESENCE VOLUME 14 NUMBER 1

world the coordinates of the avatar are affiliated withthe view scene graph rather than the object scenegraph so that the avatar always moves with the view Inother usersrsquo worlds however the avatar object is underthe object scene graphs Figure 13(a) is the scene graphof the world of a user say A who interacts with the vir-tual environment at a location Pa and with a scale SaFigure 13(b) is the scene graph of the world of anotheruser say B User Arsquos avatar is in User Brsquos object graphBecause Pa is not governed by Sa the normalized coor-dinates of the avatar in User Brsquos object graph should bewritten as

PnPaSa (4)

The body size of User Arsquos avatar changes with thescale of User A so a scale parameter is also needed todefine the size of User Arsquos avatar in User Brsquos objectscene graph so that User B can be aware of User Arsquosscale This scale should also be normalized Let Sn de-note the normalized scale where Sn is simply the recip-rocal of Sa

Sn 1Sa (5)

This is because the avatar body changes inversely pro-portional to the scale factor Sa (shrinking the world isequivalent to growing the avatar body and growing theworld is equivalent to shrinking the avatar body)

The implementation of other cross-scale tools includ-ing direct avatar manipulation also uses Pn and Sn todefine the coordinates and size of an avatar entity Thisapproach guarantees the consistency of the avatar posi-tion and size in all usersrsquo views at different scales

5 Experimental Evaluation

We evaluated multiscale collaboration in subjecttests An experiment was designed to assess the benefitsusers can get from an mCVE in accomplishing tasks re-quiring cross-scale coordination

51 Experiment Design and Procedure

Subject tasks in the experiment involved searchingfor a ldquobombrdquo on a square ground plane (2000 2000m2) with a distinctly shaped building in each corner(square hexagon octagon and circle as seen fromabove) Each had a height of 12 m and a base of about6400 m2 On the ground behind each building was aunit cube (1 m3) containing a unique text name andsmiley face (about 05 05 m2) The bomb was insideone of these four cubes In the test subjects were placedin the middle of the square plane They had a defaulteye level of 168 m and a default moving step of 1 m inthe virtual environment The shape of the building thatthe bomb was nearby was known in advance and sub-jects needed to find that building locate the bomb boxand key in the name of the box to defuse the bomb

A 2 2 2 factorial design that has six treatmentstwo noncollaboration and four collaboration wasadopted (Table 1) For the two noncollaboration treat-ments VE is just a conventional 3D virtual environ-ment and M-VE is a VE enhanced by multiscale toolsspecifically it allows users to change their interactionscales eye level and speed Among four collaborationtreatments CVE a conventional collaborative virtualenvironment is the only one without multiscale toolsThe other three treatments all equipped with multiscale

Figure 13 Two scene graphs for the same avatar

Table 1 Six Treatments in a 2 2 2 Design

Noncollaboration Collaboration

Nonmultiscale VE CVEMultiscale M-VE NR

GUIDEMOVE

Zhang and Furnas 43

tools differ in the assignment of subject task roles (be-ing a giant or a normal person) and in the way subjectsaffect each otherrsquos work across scales In one treatmentthe roles of subjects are not predefined and subjects canchoose their own interaction scales as desired This envi-ronment is labeled as NR (No Role) The other twomCVEs both assign one subject to be a giant and theother a normal person so that subjects can change theirinteraction scales only within a limited range In onesuch environment the giant is permitted to move thepartner directly and this treatment is labeled as MOVEAnother condition allows the giant to guide the move-ment of the partner only verbally and is denoted asGUIDE

Recruited through email 24 students paired in 12groups participated in the experiment In noncollabora-tion treatments they worked on their own and in col-laboration treatments they communicated through anaudio channel The performances were measured bytask-completion time

52 Results

An ANOVA analysis of data from four treatments(VE M-VE CVE and MOVE) shows main effects ofboth collaboration (F170 1298 p 0001) andmultiscale (F170 7090 p 00001) Interaction isnot significant (F170 187 p 176) as seen in Fig-ure 14 Subjects performed best in MOVE where theycould take full advantage of multiscale and collabora-tion and they did worst in VE where there was no helpat all

Subjects used different strategies in the different con-ditions Without multiscale tools subjects had to goaround and count the number of sides of all four build-ings to find the target a very time-consuming processWith multiscale subjects could increase their sizes to seebuilding shapes from above and approach the bombquickly so the time required was be reduced signifi-cantly

The performance difference among the three mCVEsalso indicates the importance of different collaborationsupports In the NR treatment subjects had to assumetheir roles via an expensive negotiation process In the

GUIDE treatment the giant subject who could see thebuilding shape must coordinate with the partner aboutnavigation verbally through a costly grounding process(Clark amp Brennan 1991) In MOVE however the all-seeing giant could actually move the partner to the des-tination quickly and bypassed grounding processes

6 Discussion

In this paper we have introduced a novel interac-tive environment multiscale collaborative virtual envi-ronment which gives users the freedom to control theirinteraction scales in collaboration Multiscale collabora-tion can help better allocate scarce scale resources andcombine different expertise and action capabilities ofindividual users in their work with large structures Wehave also discussed the implications of multiscale tech-nology for user interaction and the design and develop-ment of a Java-based mCVE system Our subject testshave shown that multiscale collaboration can facilitatepeoplersquos work that requires cross-scale information andcollaboration

Research reported in this paper has largely focused ongeneral interaction issues in mCVEs Designs have beenlimited to generic tools to support multiscale collabora-tion and experimental study has focused on generaltasks We expect that multiscale collaboration tools canbe used to support peoplersquos specific work in the realworld To achieve this goal the design of multiscale

Figure 14 Time (in seconds) for the six treatments

44 PRESENCE VOLUME 14 NUMBER 1

tools should support domain-specific tasks One of ourongoing research projects focuses on studying how multi-scale collaboration can support research in materials sci-ence and engineering We are investigating the needs ofmaterials scientists for multiscale collaboration and willdesign and implement tools that support these needsThe effectiveness and usability of these tools will also beevaluated when they are ready

At the same time we are interested in behavior issuesin mCVEs Some important research questions concern-ing user behavior in mCVEs have not been studiedthoroughly It is still unclear when and how users canbenefit from collaborative object manipulation acrossscale whether the designed dynamic view-transitiontechnique can indeed improve cross-scale informationsharing what mechanism is needed to help users bettermanage cross-scale collaboration overheads and soforth In addition it has been found that multiscale alsobrings some new challenges in social interactions(Zhang amp Furnas 2002) and efforts are needed tostudy what can be done to address these interaction is-sues Research in this direction will deepen our under-standing of mCVEs and help to design better tools tosupport multiscale collaboration

Acknowledgments

This research was funded in part by Microsoft Research

References

Ahl V amp Allen T (1996) Hierarchy theory A vision vocab-ulary and epistemology New York Columbia UniversityPress

Alexander C Ishikawa S amp Silverstein M (1977) A pat-tern language Oxford UK Oxford University Press

Bederson B amp Hollan J (1994) Pad A zoominggraphical interface for exploring alternate interface physicsProceedings of ACM UIST rsquo94 17ndash26

Benford S Bowers J Fahlen L Greenhalgh C amp Snow-don D (1995) User embodiment in collaborative virtualenvironments Proceedings of ACM CHI rsquo95 242ndash249

Benford S Snowdon D Colebourne A OrsquoBrien J ampRodden T (1997) Informing the design of collaborativevirtual environments Proceedings of ACM GROUP rsquo97 71ndash80

Carlsson C amp Hagsand O (1993) DIVEmdashA platform formulti-user virtual environments Computer and Graphics17(6) 663ndash669

Clark H amp Brennan S (1991) Grounding in communica-tion In L B Resnick J Levine amp S D Behrend (Eds)Perspectives on socially shared cognition (pp 127ndash149)Washington DC American Psychological Association

Dix A J (1994) Computer-supported cooperative workmdashAframework In D Rosenburg amp C Hutchison (Eds) De-sign Issues in CSCW (pp 23ndash37) Berlin Springer-Verlag

EPFL Geneva IIS TNO Lancaster UCL et al (1997)Guidelines for building CVE applications COVEN ProjectRetrieved November 21 2002 from httpwwwcsuclacukstaffmslatervrCovenPublicDeliverablesdel26PDF

Fox D (1998) Tabula rasa A multi-scale user interface sys-tem Unpublished doctoral dissertation New York Univer-sity

Furnas G (1986) Generalized fisheye views Proceedings ofACM CHI rsquo86 16ndash23

Gaver W Sellen A Heath C amp Luff P (1993) One isnot enough Multiple views in a media space Proceedings ofACM INTERCHI rsquo93 335ndash341

Hollan J amp Stornetta S (1992) Beyond being there Pro-ceedings of ACM CHI rsquo92 119ndash125

Koike H amp Yoshihara H (1993) Fractal approaches forvisualizing huge hierarchies Proceedings of IEEE VL rsquo9355ndash60

Lamping J Rao R amp Pirolli P (1995) A focus contexttechnique based on hyperbolic geometry for visualizinglarge hierarchies Proceedings of ACM CHI rsquo95 401ndash408

Leigh J Johnson A amp DeFanti T (1996) CALVIN Animmersimedia design environment utilizing heterogeneousperspectives Proceedings of IEEE International Conferenceon Multimedia Computing and Systems 20ndash23

Lieberman H (1994) Powers of ten thousand Navigating inlarge information spaces Proceedings of ACM UIST rsquo9415ndash16

Mackinlay J Card S amp Robertson G (1990) Rapid con-trolled movement through a virtual 3D workspace Proceed-ings of ACM SIGGRAPH rsquo90 171ndash176

Mine M Brooks F amp Sequin C (1997) Moving objects

Zhang and Furnas 45

in space Exploiting proprioception in virtual-environmentinteraction Proceedings of ACM SIGGRAPH rsquo97 19ndash26

Morrison P amp Morrison P (1994) Powers of ten About therelative size of things in the universe and the effect of addinganother zero San Francisco W H Freeman

Parent R (2001) Computer animation Algorithms and tech-niques San Francisco Morgan-Kaufmann

Pattee H (1973) Hierarchy theory The challenge of complexsystems New York George Braziller

Perlin K amp Fox D (1993) Pad An alternative approach tothe computer interface Proceedings of ACM SIGGRAPHrsquo93 57ndash64

Pierce J Forsberg A Conway M Hong S Zeleznik Ramp Mine M (1997) Image plane interaction techniques in3D immersive environments Proceedings of the 1997 Sympo-sium on Interactive 3D Graphics 39ndash43

Poupyrev I Billinghurst M Weghorst S amp Ichikawa T(1996) The go-go interaction technique Non-linear map-ping for direct manipulation in VR Proceedings of ACMUIST rsquo96 79ndash80

Puppo E amp Scopigno R (1997) Simplification LOD multi-resolution Principles and applications Proceedings ofEUROGRAPHICS 16 (3)

Raab A amp Ruger M (1996) 3D-Zoom Interactive visual-ization of structures and relations in complex graphics InB Girod H Nieman amp H Seidel (Eds) 3D image analy-sis and synthesis (pp 125ndash132) Sankt Augustin GermanyInfix-Verlag

Robbins M (2001) Modeling and simulation of the failureand toughness of polymersubstrate bonds The second An-nual Multiscale ModelingmdashBridging the Gap Between Atom-istic and Meso Scales

Robertson G Card S amp Mackinlay J (1991) Informationvisualization using 3D interactive animation Proceedings ofACM CHI rsquo91 461ndash462

Robertson G amp Mackinlay J (1993) The document lensProceedings of ACM UIST rsquo93 101ndash110

Robertson G Mackinlay J amp Card S (1991) Cone treesAnimated 3D visualizations of hierarchical informationCommunications of the ACM 34 (2) p 89ndash194

Sarkar M amp Brown M H (1992) Graphical fisheye viewsof graphs Proceedings of ACM CHI rsquo92 83ndash91

Singhal S amp Zyda M (1999) Networked virtual environ-ments Design and implementation New York ACM Press

Snowdon D amp Jaa-Aro K (1997) A subjective virtual envi-ronment for collaborative information visualization VirtualReality Universe rsquo97 Retrieved May 23 2003 fromhttpwwwcyberedgecomvru_paperssnowdenhtm

Stappers P Gaver W amp Overbeeke C (2000) Beyond thelimits of real-time realism 2000 Chapter to appear in L JHettinger amp M W Haas (Eds) Psychological issues in thedesign and use of virtual and adaptive environment Mah-wah NJ Erlbaum

Stefik M Bobrow D Foster G Lanning S amp Tatar D(1987) WYSIWIS revised Early experiences with multiuserinterfaces ACM Transactions on Information Systems(TOIS) 5(2) 147ndash167

Stoakley R Conway M amp Pausch R (1995) Virtual real-ity on a WIM Interactive worlds in miniature Proceedingsof ACM CHI rsquo95 265ndash272

Sutherland I (1965) The ultimate display Proceedings ofIFIP Congress 2

Tromp J G (1999) Usability design for CVEs COVENProject Retrieved November 21 2002 from httpcovenlancsacuk4deliverablesdel29pdf

Waters R Anderson D Barrus J Brogan D Casey MMcKeown S et al (1997) Diamond park and SPLINESocial virtual reality with 3D animation spoken interactionand runtime extendibility Presence Teleoperators and Vir-tual Environments 6(4) 461ndash481

Watt A amp Policarpo F (2001) 3D games Animation andadvanced real-time rendering New York Addison-Wesley

Zhang X amp Furnas G (2002) Social interactions in multi-scale CVEs Proceedings of ACM CVE 2002 31ndash38

46 PRESENCE VOLUME 14 NUMBER 1

choose a path that links V0 and V1 and to set the trajec-tory of viewpoint movement view direction along thepath and observation scale along the path View-pathand view-orientation interpolation techniques are ma-ture (Parent 2001 Watt amp Policarpo 2001) so thisresearch focused only on the interpolation of view scale

A scale-interpolation function could be linear loga-rithmic or in other forms We chose a logarithmic func-tion because it gives users a constant relative rate ofchange in their views Interpolating the view scale loga-rithmically can give users a familiar experience

Assume the animation includes n frames from V0 toV1 The view scale of the ith frame (0 i n) in a log-arithmic interpolation function can be written as

Si Si S0 (1)

where

S S1

S01n

(2)

With corresponding P i and O i from interpolationfunctions for view position and view orientation theview of the ith frame Vi V(P i O i Si) can be ob-tained Figure 12 shows two views seen in Figure 7 asFrame 0 and Frame n and other interpolated viewframes (Due to space limitations we have shown onlyfour frames used for the animation) Showing theseframes successively with a reasonable frame rate can pro-duce an animation linking these two divergent views

434 Cross-Scale Data Consistency and Ava-tar Position in Direct Avatar Manipulation Al-though changing the scaling factor of the top TG in theobject scene graph simplifies the scaling operation such

an approach poses a challenge in cross-scale data consis-tency In CVEs all users usually need to see the sameobject appearing at the same spatial location In conven-tional CVEs this can be easily achieved because an ob-jectrsquos coordinates in different usersrsquo worlds are often thesame In mCVEs however when users are at differentscales their virtual worlds are rescaled according to thescaling factors of the top TG in their own object scenegraphs The same object will have different local coordi-nates in different usersrsquo worlds When the object ismoved by a user the objectrsquos local coordinates in thisuserrsquos world cannot be used to update the objectrsquos newposition in all other usersrsquo worlds

In the implementation we used normalized coordi-nates which are calculated as the ratio of the object lo-cal coordinates in the userrsquos world to the userrsquos scaleLet Pl(xl yl zl) and Sl denote the local coordinates andthe userrsquos scale respectively The normalized coordi-nates Pn can be written as

Pn PlSl (3)

By doing so the same object will always have the samePn This Pn is used to define the coordinates of the ob-ject in every userrsquos object scene graph Modifying thespatial location of an object in a userrsquos world leads to anew Pn All users can use this Pn to update the coordi-nates of the object in their object scene graphs There-fore Pn enforces data consistency across scale Becausethe scaling factor is on the top of the scene graph theobjectrsquos final location in a userrsquos world is Pl the productof Pn and this userrsquos scale Sl With different Sl users willbe able to see the world rendered at different scales

An avatarrsquos position can be managed similarly butwith a small difference An avatar object differs fromother ordinary objects in a way that in its host userrsquos

Figure 11 Scale-based semantic representation node

Figure 12 Frames of a dynamic view-transition animation

42 PRESENCE VOLUME 14 NUMBER 1

world the coordinates of the avatar are affiliated withthe view scene graph rather than the object scenegraph so that the avatar always moves with the view Inother usersrsquo worlds however the avatar object is underthe object scene graphs Figure 13(a) is the scene graphof the world of a user say A who interacts with the vir-tual environment at a location Pa and with a scale SaFigure 13(b) is the scene graph of the world of anotheruser say B User Arsquos avatar is in User Brsquos object graphBecause Pa is not governed by Sa the normalized coor-dinates of the avatar in User Brsquos object graph should bewritten as

PnPaSa (4)

The body size of User Arsquos avatar changes with thescale of User A so a scale parameter is also needed todefine the size of User Arsquos avatar in User Brsquos objectscene graph so that User B can be aware of User Arsquosscale This scale should also be normalized Let Sn de-note the normalized scale where Sn is simply the recip-rocal of Sa

Sn 1Sa (5)

This is because the avatar body changes inversely pro-portional to the scale factor Sa (shrinking the world isequivalent to growing the avatar body and growing theworld is equivalent to shrinking the avatar body)

The implementation of other cross-scale tools includ-ing direct avatar manipulation also uses Pn and Sn todefine the coordinates and size of an avatar entity Thisapproach guarantees the consistency of the avatar posi-tion and size in all usersrsquo views at different scales

5 Experimental Evaluation

We evaluated multiscale collaboration in subjecttests An experiment was designed to assess the benefitsusers can get from an mCVE in accomplishing tasks re-quiring cross-scale coordination

51 Experiment Design and Procedure

Subject tasks in the experiment involved searchingfor a ldquobombrdquo on a square ground plane (2000 2000m2) with a distinctly shaped building in each corner(square hexagon octagon and circle as seen fromabove) Each had a height of 12 m and a base of about6400 m2 On the ground behind each building was aunit cube (1 m3) containing a unique text name andsmiley face (about 05 05 m2) The bomb was insideone of these four cubes In the test subjects were placedin the middle of the square plane They had a defaulteye level of 168 m and a default moving step of 1 m inthe virtual environment The shape of the building thatthe bomb was nearby was known in advance and sub-jects needed to find that building locate the bomb boxand key in the name of the box to defuse the bomb

A 2 2 2 factorial design that has six treatmentstwo noncollaboration and four collaboration wasadopted (Table 1) For the two noncollaboration treat-ments VE is just a conventional 3D virtual environ-ment and M-VE is a VE enhanced by multiscale toolsspecifically it allows users to change their interactionscales eye level and speed Among four collaborationtreatments CVE a conventional collaborative virtualenvironment is the only one without multiscale toolsThe other three treatments all equipped with multiscale

Figure 13 Two scene graphs for the same avatar

Table 1 Six Treatments in a 2 2 2 Design

Noncollaboration Collaboration

Nonmultiscale VE CVEMultiscale M-VE NR

GUIDEMOVE

Zhang and Furnas 43

tools differ in the assignment of subject task roles (be-ing a giant or a normal person) and in the way subjectsaffect each otherrsquos work across scales In one treatmentthe roles of subjects are not predefined and subjects canchoose their own interaction scales as desired This envi-ronment is labeled as NR (No Role) The other twomCVEs both assign one subject to be a giant and theother a normal person so that subjects can change theirinteraction scales only within a limited range In onesuch environment the giant is permitted to move thepartner directly and this treatment is labeled as MOVEAnother condition allows the giant to guide the move-ment of the partner only verbally and is denoted asGUIDE

Recruited through email 24 students paired in 12groups participated in the experiment In noncollabora-tion treatments they worked on their own and in col-laboration treatments they communicated through anaudio channel The performances were measured bytask-completion time

52 Results

An ANOVA analysis of data from four treatments(VE M-VE CVE and MOVE) shows main effects ofboth collaboration (F170 1298 p 0001) andmultiscale (F170 7090 p 00001) Interaction isnot significant (F170 187 p 176) as seen in Fig-ure 14 Subjects performed best in MOVE where theycould take full advantage of multiscale and collabora-tion and they did worst in VE where there was no helpat all

Subjects used different strategies in the different con-ditions Without multiscale tools subjects had to goaround and count the number of sides of all four build-ings to find the target a very time-consuming processWith multiscale subjects could increase their sizes to seebuilding shapes from above and approach the bombquickly so the time required was be reduced signifi-cantly

The performance difference among the three mCVEsalso indicates the importance of different collaborationsupports In the NR treatment subjects had to assumetheir roles via an expensive negotiation process In the

GUIDE treatment the giant subject who could see thebuilding shape must coordinate with the partner aboutnavigation verbally through a costly grounding process(Clark amp Brennan 1991) In MOVE however the all-seeing giant could actually move the partner to the des-tination quickly and bypassed grounding processes

6 Discussion

In this paper we have introduced a novel interac-tive environment multiscale collaborative virtual envi-ronment which gives users the freedom to control theirinteraction scales in collaboration Multiscale collabora-tion can help better allocate scarce scale resources andcombine different expertise and action capabilities ofindividual users in their work with large structures Wehave also discussed the implications of multiscale tech-nology for user interaction and the design and develop-ment of a Java-based mCVE system Our subject testshave shown that multiscale collaboration can facilitatepeoplersquos work that requires cross-scale information andcollaboration

Research reported in this paper has largely focused ongeneral interaction issues in mCVEs Designs have beenlimited to generic tools to support multiscale collabora-tion and experimental study has focused on generaltasks We expect that multiscale collaboration tools canbe used to support peoplersquos specific work in the realworld To achieve this goal the design of multiscale

Figure 14 Time (in seconds) for the six treatments

44 PRESENCE VOLUME 14 NUMBER 1

tools should support domain-specific tasks One of ourongoing research projects focuses on studying how multi-scale collaboration can support research in materials sci-ence and engineering We are investigating the needs ofmaterials scientists for multiscale collaboration and willdesign and implement tools that support these needsThe effectiveness and usability of these tools will also beevaluated when they are ready

At the same time we are interested in behavior issuesin mCVEs Some important research questions concern-ing user behavior in mCVEs have not been studiedthoroughly It is still unclear when and how users canbenefit from collaborative object manipulation acrossscale whether the designed dynamic view-transitiontechnique can indeed improve cross-scale informationsharing what mechanism is needed to help users bettermanage cross-scale collaboration overheads and soforth In addition it has been found that multiscale alsobrings some new challenges in social interactions(Zhang amp Furnas 2002) and efforts are needed tostudy what can be done to address these interaction is-sues Research in this direction will deepen our under-standing of mCVEs and help to design better tools tosupport multiscale collaboration

Acknowledgments

This research was funded in part by Microsoft Research

References

Ahl V amp Allen T (1996) Hierarchy theory A vision vocab-ulary and epistemology New York Columbia UniversityPress

Alexander C Ishikawa S amp Silverstein M (1977) A pat-tern language Oxford UK Oxford University Press

Bederson B amp Hollan J (1994) Pad A zoominggraphical interface for exploring alternate interface physicsProceedings of ACM UIST rsquo94 17ndash26

Benford S Bowers J Fahlen L Greenhalgh C amp Snow-don D (1995) User embodiment in collaborative virtualenvironments Proceedings of ACM CHI rsquo95 242ndash249

Benford S Snowdon D Colebourne A OrsquoBrien J ampRodden T (1997) Informing the design of collaborativevirtual environments Proceedings of ACM GROUP rsquo97 71ndash80

Carlsson C amp Hagsand O (1993) DIVEmdashA platform formulti-user virtual environments Computer and Graphics17(6) 663ndash669

Clark H amp Brennan S (1991) Grounding in communica-tion In L B Resnick J Levine amp S D Behrend (Eds)Perspectives on socially shared cognition (pp 127ndash149)Washington DC American Psychological Association

Dix A J (1994) Computer-supported cooperative workmdashAframework In D Rosenburg amp C Hutchison (Eds) De-sign Issues in CSCW (pp 23ndash37) Berlin Springer-Verlag

EPFL Geneva IIS TNO Lancaster UCL et al (1997)Guidelines for building CVE applications COVEN ProjectRetrieved November 21 2002 from httpwwwcsuclacukstaffmslatervrCovenPublicDeliverablesdel26PDF

Fox D (1998) Tabula rasa A multi-scale user interface sys-tem Unpublished doctoral dissertation New York Univer-sity

Furnas G (1986) Generalized fisheye views Proceedings ofACM CHI rsquo86 16ndash23

Gaver W Sellen A Heath C amp Luff P (1993) One isnot enough Multiple views in a media space Proceedings ofACM INTERCHI rsquo93 335ndash341

Hollan J amp Stornetta S (1992) Beyond being there Pro-ceedings of ACM CHI rsquo92 119ndash125

Koike H amp Yoshihara H (1993) Fractal approaches forvisualizing huge hierarchies Proceedings of IEEE VL rsquo9355ndash60

Lamping J Rao R amp Pirolli P (1995) A focus contexttechnique based on hyperbolic geometry for visualizinglarge hierarchies Proceedings of ACM CHI rsquo95 401ndash408

Leigh J Johnson A amp DeFanti T (1996) CALVIN Animmersimedia design environment utilizing heterogeneousperspectives Proceedings of IEEE International Conferenceon Multimedia Computing and Systems 20ndash23

Lieberman H (1994) Powers of ten thousand Navigating inlarge information spaces Proceedings of ACM UIST rsquo9415ndash16

Mackinlay J Card S amp Robertson G (1990) Rapid con-trolled movement through a virtual 3D workspace Proceed-ings of ACM SIGGRAPH rsquo90 171ndash176

Mine M Brooks F amp Sequin C (1997) Moving objects

Zhang and Furnas 45

in space Exploiting proprioception in virtual-environmentinteraction Proceedings of ACM SIGGRAPH rsquo97 19ndash26

Morrison P amp Morrison P (1994) Powers of ten About therelative size of things in the universe and the effect of addinganother zero San Francisco W H Freeman

Parent R (2001) Computer animation Algorithms and tech-niques San Francisco Morgan-Kaufmann

Pattee H (1973) Hierarchy theory The challenge of complexsystems New York George Braziller

Perlin K amp Fox D (1993) Pad An alternative approach tothe computer interface Proceedings of ACM SIGGRAPHrsquo93 57ndash64

Pierce J Forsberg A Conway M Hong S Zeleznik Ramp Mine M (1997) Image plane interaction techniques in3D immersive environments Proceedings of the 1997 Sympo-sium on Interactive 3D Graphics 39ndash43

Poupyrev I Billinghurst M Weghorst S amp Ichikawa T(1996) The go-go interaction technique Non-linear map-ping for direct manipulation in VR Proceedings of ACMUIST rsquo96 79ndash80

Puppo E amp Scopigno R (1997) Simplification LOD multi-resolution Principles and applications Proceedings ofEUROGRAPHICS 16 (3)

Raab A amp Ruger M (1996) 3D-Zoom Interactive visual-ization of structures and relations in complex graphics InB Girod H Nieman amp H Seidel (Eds) 3D image analy-sis and synthesis (pp 125ndash132) Sankt Augustin GermanyInfix-Verlag

Robbins M (2001) Modeling and simulation of the failureand toughness of polymersubstrate bonds The second An-nual Multiscale ModelingmdashBridging the Gap Between Atom-istic and Meso Scales

Robertson G Card S amp Mackinlay J (1991) Informationvisualization using 3D interactive animation Proceedings ofACM CHI rsquo91 461ndash462

Robertson G amp Mackinlay J (1993) The document lensProceedings of ACM UIST rsquo93 101ndash110

Robertson G Mackinlay J amp Card S (1991) Cone treesAnimated 3D visualizations of hierarchical informationCommunications of the ACM 34 (2) p 89ndash194

Sarkar M amp Brown M H (1992) Graphical fisheye viewsof graphs Proceedings of ACM CHI rsquo92 83ndash91

Singhal S amp Zyda M (1999) Networked virtual environ-ments Design and implementation New York ACM Press

Snowdon D amp Jaa-Aro K (1997) A subjective virtual envi-ronment for collaborative information visualization VirtualReality Universe rsquo97 Retrieved May 23 2003 fromhttpwwwcyberedgecomvru_paperssnowdenhtm

Stappers P Gaver W amp Overbeeke C (2000) Beyond thelimits of real-time realism 2000 Chapter to appear in L JHettinger amp M W Haas (Eds) Psychological issues in thedesign and use of virtual and adaptive environment Mah-wah NJ Erlbaum

Stefik M Bobrow D Foster G Lanning S amp Tatar D(1987) WYSIWIS revised Early experiences with multiuserinterfaces ACM Transactions on Information Systems(TOIS) 5(2) 147ndash167

Stoakley R Conway M amp Pausch R (1995) Virtual real-ity on a WIM Interactive worlds in miniature Proceedingsof ACM CHI rsquo95 265ndash272

Sutherland I (1965) The ultimate display Proceedings ofIFIP Congress 2

Tromp J G (1999) Usability design for CVEs COVENProject Retrieved November 21 2002 from httpcovenlancsacuk4deliverablesdel29pdf

Waters R Anderson D Barrus J Brogan D Casey MMcKeown S et al (1997) Diamond park and SPLINESocial virtual reality with 3D animation spoken interactionand runtime extendibility Presence Teleoperators and Vir-tual Environments 6(4) 461ndash481

Watt A amp Policarpo F (2001) 3D games Animation andadvanced real-time rendering New York Addison-Wesley

Zhang X amp Furnas G (2002) Social interactions in multi-scale CVEs Proceedings of ACM CVE 2002 31ndash38

46 PRESENCE VOLUME 14 NUMBER 1

world the coordinates of the avatar are affiliated withthe view scene graph rather than the object scenegraph so that the avatar always moves with the view Inother usersrsquo worlds however the avatar object is underthe object scene graphs Figure 13(a) is the scene graphof the world of a user say A who interacts with the vir-tual environment at a location Pa and with a scale SaFigure 13(b) is the scene graph of the world of anotheruser say B User Arsquos avatar is in User Brsquos object graphBecause Pa is not governed by Sa the normalized coor-dinates of the avatar in User Brsquos object graph should bewritten as

PnPaSa (4)

The body size of User Arsquos avatar changes with thescale of User A so a scale parameter is also needed todefine the size of User Arsquos avatar in User Brsquos objectscene graph so that User B can be aware of User Arsquosscale This scale should also be normalized Let Sn de-note the normalized scale where Sn is simply the recip-rocal of Sa

Sn 1Sa (5)

This is because the avatar body changes inversely pro-portional to the scale factor Sa (shrinking the world isequivalent to growing the avatar body and growing theworld is equivalent to shrinking the avatar body)

The implementation of other cross-scale tools includ-ing direct avatar manipulation also uses Pn and Sn todefine the coordinates and size of an avatar entity Thisapproach guarantees the consistency of the avatar posi-tion and size in all usersrsquo views at different scales

5 Experimental Evaluation

We evaluated multiscale collaboration in subjecttests An experiment was designed to assess the benefitsusers can get from an mCVE in accomplishing tasks re-quiring cross-scale coordination

51 Experiment Design and Procedure

Subject tasks in the experiment involved searchingfor a ldquobombrdquo on a square ground plane (2000 2000m2) with a distinctly shaped building in each corner(square hexagon octagon and circle as seen fromabove) Each had a height of 12 m and a base of about6400 m2 On the ground behind each building was aunit cube (1 m3) containing a unique text name andsmiley face (about 05 05 m2) The bomb was insideone of these four cubes In the test subjects were placedin the middle of the square plane They had a defaulteye level of 168 m and a default moving step of 1 m inthe virtual environment The shape of the building thatthe bomb was nearby was known in advance and sub-jects needed to find that building locate the bomb boxand key in the name of the box to defuse the bomb

A 2 2 2 factorial design that has six treatmentstwo noncollaboration and four collaboration wasadopted (Table 1) For the two noncollaboration treat-ments VE is just a conventional 3D virtual environ-ment and M-VE is a VE enhanced by multiscale toolsspecifically it allows users to change their interactionscales eye level and speed Among four collaborationtreatments CVE a conventional collaborative virtualenvironment is the only one without multiscale toolsThe other three treatments all equipped with multiscale

Figure 13 Two scene graphs for the same avatar

Table 1 Six Treatments in a 2 2 2 Design

Noncollaboration Collaboration

Nonmultiscale VE CVEMultiscale M-VE NR

GUIDEMOVE

Zhang and Furnas 43

tools differ in the assignment of subject task roles (be-ing a giant or a normal person) and in the way subjectsaffect each otherrsquos work across scales In one treatmentthe roles of subjects are not predefined and subjects canchoose their own interaction scales as desired This envi-ronment is labeled as NR (No Role) The other twomCVEs both assign one subject to be a giant and theother a normal person so that subjects can change theirinteraction scales only within a limited range In onesuch environment the giant is permitted to move thepartner directly and this treatment is labeled as MOVEAnother condition allows the giant to guide the move-ment of the partner only verbally and is denoted asGUIDE

Recruited through email 24 students paired in 12groups participated in the experiment In noncollabora-tion treatments they worked on their own and in col-laboration treatments they communicated through anaudio channel The performances were measured bytask-completion time

52 Results

An ANOVA analysis of data from four treatments(VE M-VE CVE and MOVE) shows main effects ofboth collaboration (F170 1298 p 0001) andmultiscale (F170 7090 p 00001) Interaction isnot significant (F170 187 p 176) as seen in Fig-ure 14 Subjects performed best in MOVE where theycould take full advantage of multiscale and collabora-tion and they did worst in VE where there was no helpat all

Subjects used different strategies in the different con-ditions Without multiscale tools subjects had to goaround and count the number of sides of all four build-ings to find the target a very time-consuming processWith multiscale subjects could increase their sizes to seebuilding shapes from above and approach the bombquickly so the time required was be reduced signifi-cantly

The performance difference among the three mCVEsalso indicates the importance of different collaborationsupports In the NR treatment subjects had to assumetheir roles via an expensive negotiation process In the

GUIDE treatment the giant subject who could see thebuilding shape must coordinate with the partner aboutnavigation verbally through a costly grounding process(Clark amp Brennan 1991) In MOVE however the all-seeing giant could actually move the partner to the des-tination quickly and bypassed grounding processes

6 Discussion

In this paper we have introduced a novel interac-tive environment multiscale collaborative virtual envi-ronment which gives users the freedom to control theirinteraction scales in collaboration Multiscale collabora-tion can help better allocate scarce scale resources andcombine different expertise and action capabilities ofindividual users in their work with large structures Wehave also discussed the implications of multiscale tech-nology for user interaction and the design and develop-ment of a Java-based mCVE system Our subject testshave shown that multiscale collaboration can facilitatepeoplersquos work that requires cross-scale information andcollaboration

Research reported in this paper has largely focused ongeneral interaction issues in mCVEs Designs have beenlimited to generic tools to support multiscale collabora-tion and experimental study has focused on generaltasks We expect that multiscale collaboration tools canbe used to support peoplersquos specific work in the realworld To achieve this goal the design of multiscale

Figure 14 Time (in seconds) for the six treatments

44 PRESENCE VOLUME 14 NUMBER 1

tools should support domain-specific tasks One of ourongoing research projects focuses on studying how multi-scale collaboration can support research in materials sci-ence and engineering We are investigating the needs ofmaterials scientists for multiscale collaboration and willdesign and implement tools that support these needsThe effectiveness and usability of these tools will also beevaluated when they are ready

At the same time we are interested in behavior issuesin mCVEs Some important research questions concern-ing user behavior in mCVEs have not been studiedthoroughly It is still unclear when and how users canbenefit from collaborative object manipulation acrossscale whether the designed dynamic view-transitiontechnique can indeed improve cross-scale informationsharing what mechanism is needed to help users bettermanage cross-scale collaboration overheads and soforth In addition it has been found that multiscale alsobrings some new challenges in social interactions(Zhang amp Furnas 2002) and efforts are needed tostudy what can be done to address these interaction is-sues Research in this direction will deepen our under-standing of mCVEs and help to design better tools tosupport multiscale collaboration

Acknowledgments

This research was funded in part by Microsoft Research

References

Ahl V amp Allen T (1996) Hierarchy theory A vision vocab-ulary and epistemology New York Columbia UniversityPress

Alexander C Ishikawa S amp Silverstein M (1977) A pat-tern language Oxford UK Oxford University Press

Bederson B amp Hollan J (1994) Pad A zoominggraphical interface for exploring alternate interface physicsProceedings of ACM UIST rsquo94 17ndash26

Benford S Bowers J Fahlen L Greenhalgh C amp Snow-don D (1995) User embodiment in collaborative virtualenvironments Proceedings of ACM CHI rsquo95 242ndash249

Benford S Snowdon D Colebourne A OrsquoBrien J ampRodden T (1997) Informing the design of collaborativevirtual environments Proceedings of ACM GROUP rsquo97 71ndash80

Carlsson C amp Hagsand O (1993) DIVEmdashA platform formulti-user virtual environments Computer and Graphics17(6) 663ndash669

Clark H amp Brennan S (1991) Grounding in communica-tion In L B Resnick J Levine amp S D Behrend (Eds)Perspectives on socially shared cognition (pp 127ndash149)Washington DC American Psychological Association

Dix A J (1994) Computer-supported cooperative workmdashAframework In D Rosenburg amp C Hutchison (Eds) De-sign Issues in CSCW (pp 23ndash37) Berlin Springer-Verlag

EPFL Geneva IIS TNO Lancaster UCL et al (1997)Guidelines for building CVE applications COVEN ProjectRetrieved November 21 2002 from httpwwwcsuclacukstaffmslatervrCovenPublicDeliverablesdel26PDF

Fox D (1998) Tabula rasa A multi-scale user interface sys-tem Unpublished doctoral dissertation New York Univer-sity

Furnas G (1986) Generalized fisheye views Proceedings ofACM CHI rsquo86 16ndash23

Gaver W Sellen A Heath C amp Luff P (1993) One isnot enough Multiple views in a media space Proceedings ofACM INTERCHI rsquo93 335ndash341

Hollan J amp Stornetta S (1992) Beyond being there Pro-ceedings of ACM CHI rsquo92 119ndash125

Koike H amp Yoshihara H (1993) Fractal approaches forvisualizing huge hierarchies Proceedings of IEEE VL rsquo9355ndash60

Lamping J Rao R amp Pirolli P (1995) A focus contexttechnique based on hyperbolic geometry for visualizinglarge hierarchies Proceedings of ACM CHI rsquo95 401ndash408

Leigh J Johnson A amp DeFanti T (1996) CALVIN Animmersimedia design environment utilizing heterogeneousperspectives Proceedings of IEEE International Conferenceon Multimedia Computing and Systems 20ndash23

Lieberman H (1994) Powers of ten thousand Navigating inlarge information spaces Proceedings of ACM UIST rsquo9415ndash16

Mackinlay J Card S amp Robertson G (1990) Rapid con-trolled movement through a virtual 3D workspace Proceed-ings of ACM SIGGRAPH rsquo90 171ndash176

Mine M Brooks F amp Sequin C (1997) Moving objects

Zhang and Furnas 45

in space Exploiting proprioception in virtual-environmentinteraction Proceedings of ACM SIGGRAPH rsquo97 19ndash26

Morrison P amp Morrison P (1994) Powers of ten About therelative size of things in the universe and the effect of addinganother zero San Francisco W H Freeman

Parent R (2001) Computer animation Algorithms and tech-niques San Francisco Morgan-Kaufmann

Pattee H (1973) Hierarchy theory The challenge of complexsystems New York George Braziller

Perlin K amp Fox D (1993) Pad An alternative approach tothe computer interface Proceedings of ACM SIGGRAPHrsquo93 57ndash64

Pierce J Forsberg A Conway M Hong S Zeleznik Ramp Mine M (1997) Image plane interaction techniques in3D immersive environments Proceedings of the 1997 Sympo-sium on Interactive 3D Graphics 39ndash43

Poupyrev I Billinghurst M Weghorst S amp Ichikawa T(1996) The go-go interaction technique Non-linear map-ping for direct manipulation in VR Proceedings of ACMUIST rsquo96 79ndash80

Puppo E amp Scopigno R (1997) Simplification LOD multi-resolution Principles and applications Proceedings ofEUROGRAPHICS 16 (3)

Raab A amp Ruger M (1996) 3D-Zoom Interactive visual-ization of structures and relations in complex graphics InB Girod H Nieman amp H Seidel (Eds) 3D image analy-sis and synthesis (pp 125ndash132) Sankt Augustin GermanyInfix-Verlag

Robbins M (2001) Modeling and simulation of the failureand toughness of polymersubstrate bonds The second An-nual Multiscale ModelingmdashBridging the Gap Between Atom-istic and Meso Scales

Robertson G Card S amp Mackinlay J (1991) Informationvisualization using 3D interactive animation Proceedings ofACM CHI rsquo91 461ndash462

Robertson G amp Mackinlay J (1993) The document lensProceedings of ACM UIST rsquo93 101ndash110

Robertson G Mackinlay J amp Card S (1991) Cone treesAnimated 3D visualizations of hierarchical informationCommunications of the ACM 34 (2) p 89ndash194

Sarkar M amp Brown M H (1992) Graphical fisheye viewsof graphs Proceedings of ACM CHI rsquo92 83ndash91

Singhal S amp Zyda M (1999) Networked virtual environ-ments Design and implementation New York ACM Press

Snowdon D amp Jaa-Aro K (1997) A subjective virtual envi-ronment for collaborative information visualization VirtualReality Universe rsquo97 Retrieved May 23 2003 fromhttpwwwcyberedgecomvru_paperssnowdenhtm

Stappers P Gaver W amp Overbeeke C (2000) Beyond thelimits of real-time realism 2000 Chapter to appear in L JHettinger amp M W Haas (Eds) Psychological issues in thedesign and use of virtual and adaptive environment Mah-wah NJ Erlbaum

Stefik M Bobrow D Foster G Lanning S amp Tatar D(1987) WYSIWIS revised Early experiences with multiuserinterfaces ACM Transactions on Information Systems(TOIS) 5(2) 147ndash167

Stoakley R Conway M amp Pausch R (1995) Virtual real-ity on a WIM Interactive worlds in miniature Proceedingsof ACM CHI rsquo95 265ndash272

Sutherland I (1965) The ultimate display Proceedings ofIFIP Congress 2

Tromp J G (1999) Usability design for CVEs COVENProject Retrieved November 21 2002 from httpcovenlancsacuk4deliverablesdel29pdf

Waters R Anderson D Barrus J Brogan D Casey MMcKeown S et al (1997) Diamond park and SPLINESocial virtual reality with 3D animation spoken interactionand runtime extendibility Presence Teleoperators and Vir-tual Environments 6(4) 461ndash481

Watt A amp Policarpo F (2001) 3D games Animation andadvanced real-time rendering New York Addison-Wesley

Zhang X amp Furnas G (2002) Social interactions in multi-scale CVEs Proceedings of ACM CVE 2002 31ndash38

46 PRESENCE VOLUME 14 NUMBER 1

tools differ in the assignment of subject task roles (be-ing a giant or a normal person) and in the way subjectsaffect each otherrsquos work across scales In one treatmentthe roles of subjects are not predefined and subjects canchoose their own interaction scales as desired This envi-ronment is labeled as NR (No Role) The other twomCVEs both assign one subject to be a giant and theother a normal person so that subjects can change theirinteraction scales only within a limited range In onesuch environment the giant is permitted to move thepartner directly and this treatment is labeled as MOVEAnother condition allows the giant to guide the move-ment of the partner only verbally and is denoted asGUIDE

Recruited through email 24 students paired in 12groups participated in the experiment In noncollabora-tion treatments they worked on their own and in col-laboration treatments they communicated through anaudio channel The performances were measured bytask-completion time

52 Results

An ANOVA analysis of data from four treatments(VE M-VE CVE and MOVE) shows main effects ofboth collaboration (F170 1298 p 0001) andmultiscale (F170 7090 p 00001) Interaction isnot significant (F170 187 p 176) as seen in Fig-ure 14 Subjects performed best in MOVE where theycould take full advantage of multiscale and collabora-tion and they did worst in VE where there was no helpat all

Subjects used different strategies in the different con-ditions Without multiscale tools subjects had to goaround and count the number of sides of all four build-ings to find the target a very time-consuming processWith multiscale subjects could increase their sizes to seebuilding shapes from above and approach the bombquickly so the time required was be reduced signifi-cantly

The performance difference among the three mCVEsalso indicates the importance of different collaborationsupports In the NR treatment subjects had to assumetheir roles via an expensive negotiation process In the

GUIDE treatment the giant subject who could see thebuilding shape must coordinate with the partner aboutnavigation verbally through a costly grounding process(Clark amp Brennan 1991) In MOVE however the all-seeing giant could actually move the partner to the des-tination quickly and bypassed grounding processes

6 Discussion

In this paper we have introduced a novel interac-tive environment multiscale collaborative virtual envi-ronment which gives users the freedom to control theirinteraction scales in collaboration Multiscale collabora-tion can help better allocate scarce scale resources andcombine different expertise and action capabilities ofindividual users in their work with large structures Wehave also discussed the implications of multiscale tech-nology for user interaction and the design and develop-ment of a Java-based mCVE system Our subject testshave shown that multiscale collaboration can facilitatepeoplersquos work that requires cross-scale information andcollaboration

Research reported in this paper has largely focused ongeneral interaction issues in mCVEs Designs have beenlimited to generic tools to support multiscale collabora-tion and experimental study has focused on generaltasks We expect that multiscale collaboration tools canbe used to support peoplersquos specific work in the realworld To achieve this goal the design of multiscale

Figure 14 Time (in seconds) for the six treatments

44 PRESENCE VOLUME 14 NUMBER 1

tools should support domain-specific tasks One of ourongoing research projects focuses on studying how multi-scale collaboration can support research in materials sci-ence and engineering We are investigating the needs ofmaterials scientists for multiscale collaboration and willdesign and implement tools that support these needsThe effectiveness and usability of these tools will also beevaluated when they are ready

At the same time we are interested in behavior issuesin mCVEs Some important research questions concern-ing user behavior in mCVEs have not been studiedthoroughly It is still unclear when and how users canbenefit from collaborative object manipulation acrossscale whether the designed dynamic view-transitiontechnique can indeed improve cross-scale informationsharing what mechanism is needed to help users bettermanage cross-scale collaboration overheads and soforth In addition it has been found that multiscale alsobrings some new challenges in social interactions(Zhang amp Furnas 2002) and efforts are needed tostudy what can be done to address these interaction is-sues Research in this direction will deepen our under-standing of mCVEs and help to design better tools tosupport multiscale collaboration

Acknowledgments

This research was funded in part by Microsoft Research

References

Ahl V amp Allen T (1996) Hierarchy theory A vision vocab-ulary and epistemology New York Columbia UniversityPress

Alexander C Ishikawa S amp Silverstein M (1977) A pat-tern language Oxford UK Oxford University Press

Bederson B amp Hollan J (1994) Pad A zoominggraphical interface for exploring alternate interface physicsProceedings of ACM UIST rsquo94 17ndash26

Benford S Bowers J Fahlen L Greenhalgh C amp Snow-don D (1995) User embodiment in collaborative virtualenvironments Proceedings of ACM CHI rsquo95 242ndash249

Benford S Snowdon D Colebourne A OrsquoBrien J ampRodden T (1997) Informing the design of collaborativevirtual environments Proceedings of ACM GROUP rsquo97 71ndash80

Carlsson C amp Hagsand O (1993) DIVEmdashA platform formulti-user virtual environments Computer and Graphics17(6) 663ndash669

Clark H amp Brennan S (1991) Grounding in communica-tion In L B Resnick J Levine amp S D Behrend (Eds)Perspectives on socially shared cognition (pp 127ndash149)Washington DC American Psychological Association

Dix A J (1994) Computer-supported cooperative workmdashAframework In D Rosenburg amp C Hutchison (Eds) De-sign Issues in CSCW (pp 23ndash37) Berlin Springer-Verlag

EPFL Geneva IIS TNO Lancaster UCL et al (1997)Guidelines for building CVE applications COVEN ProjectRetrieved November 21 2002 from httpwwwcsuclacukstaffmslatervrCovenPublicDeliverablesdel26PDF

Fox D (1998) Tabula rasa A multi-scale user interface sys-tem Unpublished doctoral dissertation New York Univer-sity

Furnas G (1986) Generalized fisheye views Proceedings ofACM CHI rsquo86 16ndash23

Gaver W Sellen A Heath C amp Luff P (1993) One isnot enough Multiple views in a media space Proceedings ofACM INTERCHI rsquo93 335ndash341

Hollan J amp Stornetta S (1992) Beyond being there Pro-ceedings of ACM CHI rsquo92 119ndash125

Koike H amp Yoshihara H (1993) Fractal approaches forvisualizing huge hierarchies Proceedings of IEEE VL rsquo9355ndash60

Lamping J Rao R amp Pirolli P (1995) A focus contexttechnique based on hyperbolic geometry for visualizinglarge hierarchies Proceedings of ACM CHI rsquo95 401ndash408

Leigh J Johnson A amp DeFanti T (1996) CALVIN Animmersimedia design environment utilizing heterogeneousperspectives Proceedings of IEEE International Conferenceon Multimedia Computing and Systems 20ndash23

Lieberman H (1994) Powers of ten thousand Navigating inlarge information spaces Proceedings of ACM UIST rsquo9415ndash16

Mackinlay J Card S amp Robertson G (1990) Rapid con-trolled movement through a virtual 3D workspace Proceed-ings of ACM SIGGRAPH rsquo90 171ndash176

Mine M Brooks F amp Sequin C (1997) Moving objects

Zhang and Furnas 45

in space Exploiting proprioception in virtual-environmentinteraction Proceedings of ACM SIGGRAPH rsquo97 19ndash26

Morrison P amp Morrison P (1994) Powers of ten About therelative size of things in the universe and the effect of addinganother zero San Francisco W H Freeman

Parent R (2001) Computer animation Algorithms and tech-niques San Francisco Morgan-Kaufmann

Pattee H (1973) Hierarchy theory The challenge of complexsystems New York George Braziller

Perlin K amp Fox D (1993) Pad An alternative approach tothe computer interface Proceedings of ACM SIGGRAPHrsquo93 57ndash64

Pierce J Forsberg A Conway M Hong S Zeleznik Ramp Mine M (1997) Image plane interaction techniques in3D immersive environments Proceedings of the 1997 Sympo-sium on Interactive 3D Graphics 39ndash43

Poupyrev I Billinghurst M Weghorst S amp Ichikawa T(1996) The go-go interaction technique Non-linear map-ping for direct manipulation in VR Proceedings of ACMUIST rsquo96 79ndash80

Puppo E amp Scopigno R (1997) Simplification LOD multi-resolution Principles and applications Proceedings ofEUROGRAPHICS 16 (3)

Raab A amp Ruger M (1996) 3D-Zoom Interactive visual-ization of structures and relations in complex graphics InB Girod H Nieman amp H Seidel (Eds) 3D image analy-sis and synthesis (pp 125ndash132) Sankt Augustin GermanyInfix-Verlag

Robbins M (2001) Modeling and simulation of the failureand toughness of polymersubstrate bonds The second An-nual Multiscale ModelingmdashBridging the Gap Between Atom-istic and Meso Scales

Robertson G Card S amp Mackinlay J (1991) Informationvisualization using 3D interactive animation Proceedings ofACM CHI rsquo91 461ndash462

Robertson G amp Mackinlay J (1993) The document lensProceedings of ACM UIST rsquo93 101ndash110

Robertson G Mackinlay J amp Card S (1991) Cone treesAnimated 3D visualizations of hierarchical informationCommunications of the ACM 34 (2) p 89ndash194

Sarkar M amp Brown M H (1992) Graphical fisheye viewsof graphs Proceedings of ACM CHI rsquo92 83ndash91

Singhal S amp Zyda M (1999) Networked virtual environ-ments Design and implementation New York ACM Press

Snowdon D amp Jaa-Aro K (1997) A subjective virtual envi-ronment for collaborative information visualization VirtualReality Universe rsquo97 Retrieved May 23 2003 fromhttpwwwcyberedgecomvru_paperssnowdenhtm

Stappers P Gaver W amp Overbeeke C (2000) Beyond thelimits of real-time realism 2000 Chapter to appear in L JHettinger amp M W Haas (Eds) Psychological issues in thedesign and use of virtual and adaptive environment Mah-wah NJ Erlbaum

Stefik M Bobrow D Foster G Lanning S amp Tatar D(1987) WYSIWIS revised Early experiences with multiuserinterfaces ACM Transactions on Information Systems(TOIS) 5(2) 147ndash167

Stoakley R Conway M amp Pausch R (1995) Virtual real-ity on a WIM Interactive worlds in miniature Proceedingsof ACM CHI rsquo95 265ndash272

Sutherland I (1965) The ultimate display Proceedings ofIFIP Congress 2

Tromp J G (1999) Usability design for CVEs COVENProject Retrieved November 21 2002 from httpcovenlancsacuk4deliverablesdel29pdf

Waters R Anderson D Barrus J Brogan D Casey MMcKeown S et al (1997) Diamond park and SPLINESocial virtual reality with 3D animation spoken interactionand runtime extendibility Presence Teleoperators and Vir-tual Environments 6(4) 461ndash481

Watt A amp Policarpo F (2001) 3D games Animation andadvanced real-time rendering New York Addison-Wesley

Zhang X amp Furnas G (2002) Social interactions in multi-scale CVEs Proceedings of ACM CVE 2002 31ndash38

46 PRESENCE VOLUME 14 NUMBER 1

tools should support domain-specific tasks One of ourongoing research projects focuses on studying how multi-scale collaboration can support research in materials sci-ence and engineering We are investigating the needs ofmaterials scientists for multiscale collaboration and willdesign and implement tools that support these needsThe effectiveness and usability of these tools will also beevaluated when they are ready

At the same time we are interested in behavior issuesin mCVEs Some important research questions concern-ing user behavior in mCVEs have not been studiedthoroughly It is still unclear when and how users canbenefit from collaborative object manipulation acrossscale whether the designed dynamic view-transitiontechnique can indeed improve cross-scale informationsharing what mechanism is needed to help users bettermanage cross-scale collaboration overheads and soforth In addition it has been found that multiscale alsobrings some new challenges in social interactions(Zhang amp Furnas 2002) and efforts are needed tostudy what can be done to address these interaction is-sues Research in this direction will deepen our under-standing of mCVEs and help to design better tools tosupport multiscale collaboration

Acknowledgments

This research was funded in part by Microsoft Research

References

Ahl V amp Allen T (1996) Hierarchy theory A vision vocab-ulary and epistemology New York Columbia UniversityPress

Alexander C Ishikawa S amp Silverstein M (1977) A pat-tern language Oxford UK Oxford University Press

Bederson B amp Hollan J (1994) Pad A zoominggraphical interface for exploring alternate interface physicsProceedings of ACM UIST rsquo94 17ndash26

Benford S Bowers J Fahlen L Greenhalgh C amp Snow-don D (1995) User embodiment in collaborative virtualenvironments Proceedings of ACM CHI rsquo95 242ndash249

Benford S Snowdon D Colebourne A OrsquoBrien J ampRodden T (1997) Informing the design of collaborativevirtual environments Proceedings of ACM GROUP rsquo97 71ndash80

Carlsson C amp Hagsand O (1993) DIVEmdashA platform formulti-user virtual environments Computer and Graphics17(6) 663ndash669

Clark H amp Brennan S (1991) Grounding in communica-tion In L B Resnick J Levine amp S D Behrend (Eds)Perspectives on socially shared cognition (pp 127ndash149)Washington DC American Psychological Association

Dix A J (1994) Computer-supported cooperative workmdashAframework In D Rosenburg amp C Hutchison (Eds) De-sign Issues in CSCW (pp 23ndash37) Berlin Springer-Verlag

EPFL Geneva IIS TNO Lancaster UCL et al (1997)Guidelines for building CVE applications COVEN ProjectRetrieved November 21 2002 from httpwwwcsuclacukstaffmslatervrCovenPublicDeliverablesdel26PDF

Fox D (1998) Tabula rasa A multi-scale user interface sys-tem Unpublished doctoral dissertation New York Univer-sity

Furnas G (1986) Generalized fisheye views Proceedings ofACM CHI rsquo86 16ndash23

Gaver W Sellen A Heath C amp Luff P (1993) One isnot enough Multiple views in a media space Proceedings ofACM INTERCHI rsquo93 335ndash341

Hollan J amp Stornetta S (1992) Beyond being there Pro-ceedings of ACM CHI rsquo92 119ndash125

Koike H amp Yoshihara H (1993) Fractal approaches forvisualizing huge hierarchies Proceedings of IEEE VL rsquo9355ndash60

Lamping J Rao R amp Pirolli P (1995) A focus contexttechnique based on hyperbolic geometry for visualizinglarge hierarchies Proceedings of ACM CHI rsquo95 401ndash408

Leigh J Johnson A amp DeFanti T (1996) CALVIN Animmersimedia design environment utilizing heterogeneousperspectives Proceedings of IEEE International Conferenceon Multimedia Computing and Systems 20ndash23

Lieberman H (1994) Powers of ten thousand Navigating inlarge information spaces Proceedings of ACM UIST rsquo9415ndash16

Mackinlay J Card S amp Robertson G (1990) Rapid con-trolled movement through a virtual 3D workspace Proceed-ings of ACM SIGGRAPH rsquo90 171ndash176

Mine M Brooks F amp Sequin C (1997) Moving objects

Zhang and Furnas 45

in space Exploiting proprioception in virtual-environmentinteraction Proceedings of ACM SIGGRAPH rsquo97 19ndash26

Morrison P amp Morrison P (1994) Powers of ten About therelative size of things in the universe and the effect of addinganother zero San Francisco W H Freeman

Parent R (2001) Computer animation Algorithms and tech-niques San Francisco Morgan-Kaufmann

Pattee H (1973) Hierarchy theory The challenge of complexsystems New York George Braziller

Perlin K amp Fox D (1993) Pad An alternative approach tothe computer interface Proceedings of ACM SIGGRAPHrsquo93 57ndash64

Pierce J Forsberg A Conway M Hong S Zeleznik Ramp Mine M (1997) Image plane interaction techniques in3D immersive environments Proceedings of the 1997 Sympo-sium on Interactive 3D Graphics 39ndash43

Poupyrev I Billinghurst M Weghorst S amp Ichikawa T(1996) The go-go interaction technique Non-linear map-ping for direct manipulation in VR Proceedings of ACMUIST rsquo96 79ndash80

Puppo E amp Scopigno R (1997) Simplification LOD multi-resolution Principles and applications Proceedings ofEUROGRAPHICS 16 (3)

Raab A amp Ruger M (1996) 3D-Zoom Interactive visual-ization of structures and relations in complex graphics InB Girod H Nieman amp H Seidel (Eds) 3D image analy-sis and synthesis (pp 125ndash132) Sankt Augustin GermanyInfix-Verlag

Robbins M (2001) Modeling and simulation of the failureand toughness of polymersubstrate bonds The second An-nual Multiscale ModelingmdashBridging the Gap Between Atom-istic and Meso Scales

Robertson G Card S amp Mackinlay J (1991) Informationvisualization using 3D interactive animation Proceedings ofACM CHI rsquo91 461ndash462

Robertson G amp Mackinlay J (1993) The document lensProceedings of ACM UIST rsquo93 101ndash110

Robertson G Mackinlay J amp Card S (1991) Cone treesAnimated 3D visualizations of hierarchical informationCommunications of the ACM 34 (2) p 89ndash194

Sarkar M amp Brown M H (1992) Graphical fisheye viewsof graphs Proceedings of ACM CHI rsquo92 83ndash91

Singhal S amp Zyda M (1999) Networked virtual environ-ments Design and implementation New York ACM Press

Snowdon D amp Jaa-Aro K (1997) A subjective virtual envi-ronment for collaborative information visualization VirtualReality Universe rsquo97 Retrieved May 23 2003 fromhttpwwwcyberedgecomvru_paperssnowdenhtm

Stappers P Gaver W amp Overbeeke C (2000) Beyond thelimits of real-time realism 2000 Chapter to appear in L JHettinger amp M W Haas (Eds) Psychological issues in thedesign and use of virtual and adaptive environment Mah-wah NJ Erlbaum

Stefik M Bobrow D Foster G Lanning S amp Tatar D(1987) WYSIWIS revised Early experiences with multiuserinterfaces ACM Transactions on Information Systems(TOIS) 5(2) 147ndash167

Stoakley R Conway M amp Pausch R (1995) Virtual real-ity on a WIM Interactive worlds in miniature Proceedingsof ACM CHI rsquo95 265ndash272

Sutherland I (1965) The ultimate display Proceedings ofIFIP Congress 2

Tromp J G (1999) Usability design for CVEs COVENProject Retrieved November 21 2002 from httpcovenlancsacuk4deliverablesdel29pdf

Waters R Anderson D Barrus J Brogan D Casey MMcKeown S et al (1997) Diamond park and SPLINESocial virtual reality with 3D animation spoken interactionand runtime extendibility Presence Teleoperators and Vir-tual Environments 6(4) 461ndash481

Watt A amp Policarpo F (2001) 3D games Animation andadvanced real-time rendering New York Addison-Wesley

Zhang X amp Furnas G (2002) Social interactions in multi-scale CVEs Proceedings of ACM CVE 2002 31ndash38

46 PRESENCE VOLUME 14 NUMBER 1

in space Exploiting proprioception in virtual-environmentinteraction Proceedings of ACM SIGGRAPH rsquo97 19ndash26

Morrison P amp Morrison P (1994) Powers of ten About therelative size of things in the universe and the effect of addinganother zero San Francisco W H Freeman

Parent R (2001) Computer animation Algorithms and tech-niques San Francisco Morgan-Kaufmann

Pattee H (1973) Hierarchy theory The challenge of complexsystems New York George Braziller

Perlin K amp Fox D (1993) Pad An alternative approach tothe computer interface Proceedings of ACM SIGGRAPHrsquo93 57ndash64

Pierce J Forsberg A Conway M Hong S Zeleznik Ramp Mine M (1997) Image plane interaction techniques in3D immersive environments Proceedings of the 1997 Sympo-sium on Interactive 3D Graphics 39ndash43

Poupyrev I Billinghurst M Weghorst S amp Ichikawa T(1996) The go-go interaction technique Non-linear map-ping for direct manipulation in VR Proceedings of ACMUIST rsquo96 79ndash80

Puppo E amp Scopigno R (1997) Simplification LOD multi-resolution Principles and applications Proceedings ofEUROGRAPHICS 16 (3)

Raab A amp Ruger M (1996) 3D-Zoom Interactive visual-ization of structures and relations in complex graphics InB Girod H Nieman amp H Seidel (Eds) 3D image analy-sis and synthesis (pp 125ndash132) Sankt Augustin GermanyInfix-Verlag

Robbins M (2001) Modeling and simulation of the failureand toughness of polymersubstrate bonds The second An-nual Multiscale ModelingmdashBridging the Gap Between Atom-istic and Meso Scales

Robertson G Card S amp Mackinlay J (1991) Informationvisualization using 3D interactive animation Proceedings ofACM CHI rsquo91 461ndash462

Robertson G amp Mackinlay J (1993) The document lensProceedings of ACM UIST rsquo93 101ndash110

Robertson G Mackinlay J amp Card S (1991) Cone treesAnimated 3D visualizations of hierarchical informationCommunications of the ACM 34 (2) p 89ndash194

Sarkar M amp Brown M H (1992) Graphical fisheye viewsof graphs Proceedings of ACM CHI rsquo92 83ndash91

Singhal S amp Zyda M (1999) Networked virtual environ-ments Design and implementation New York ACM Press

Snowdon D amp Jaa-Aro K (1997) A subjective virtual envi-ronment for collaborative information visualization VirtualReality Universe rsquo97 Retrieved May 23 2003 fromhttpwwwcyberedgecomvru_paperssnowdenhtm

Stappers P Gaver W amp Overbeeke C (2000) Beyond thelimits of real-time realism 2000 Chapter to appear in L JHettinger amp M W Haas (Eds) Psychological issues in thedesign and use of virtual and adaptive environment Mah-wah NJ Erlbaum

Stefik M Bobrow D Foster G Lanning S amp Tatar D(1987) WYSIWIS revised Early experiences with multiuserinterfaces ACM Transactions on Information Systems(TOIS) 5(2) 147ndash167

Stoakley R Conway M amp Pausch R (1995) Virtual real-ity on a WIM Interactive worlds in miniature Proceedingsof ACM CHI rsquo95 265ndash272

Sutherland I (1965) The ultimate display Proceedings ofIFIP Congress 2

Tromp J G (1999) Usability design for CVEs COVENProject Retrieved November 21 2002 from httpcovenlancsacuk4deliverablesdel29pdf

Waters R Anderson D Barrus J Brogan D Casey MMcKeown S et al (1997) Diamond park and SPLINESocial virtual reality with 3D animation spoken interactionand runtime extendibility Presence Teleoperators and Vir-tual Environments 6(4) 461ndash481

Watt A amp Policarpo F (2001) 3D games Animation andadvanced real-time rendering New York Addison-Wesley

Zhang X amp Furnas G (2002) Social interactions in multi-scale CVEs Proceedings of ACM CVE 2002 31ndash38

46 PRESENCE VOLUME 14 NUMBER 1