Hybrid Environmental Projection Platform: An Interactive Educational Tool

Archi t e c ture , Ci ty & Informat ion Des ign (EuropIA.14)

Hybrid Environmental Projection Platform: An Interactive Educational Tool

Konstantinos-Alketas OUNGRINIS TIE Lab, TU Crete

K4 Building, University Campus, Kounoupidiana Chania, Greece, 73133 [email protected]

Marianthi LIAPI TIE Lab, TU Crete

K4 Building, University Campus, Kounoupidiana Chania, Greece, 73133

[email protected]

Marios CHRISTOULAKIS TIE Lab, TU Crete


[email protected]

Angelos KLOTHAKIS TIE Lab, TU Crete


Iasonas PATERAKIS TIE Lab, TU Crete


Natalia AVARIKIOTI TIE Lab, TU Crete


[email protected]

Marilena DIMOPOULOU TIE Lab, TU Crete


[email protected]

Mariel CREMLIS TIE Lab, TU Crete


Dimitris KOUTSOUMPAS TIE Lab, TU Crete


[email protected]

Despoina SAPOUNTZI TIE Lab, TU Crete


Abstract

The paper presents the design, development and operation of the Hybrid Environmental Projection Platform (HEPP), an interactive educational tool that involves the integration of advanced technological spatial systems inside the typical classroom. There are two driving forces guiding this project. Firstly, it is the introduction of information technology systems, tele-presence systems and robotics as contemporary pedagogical tools. The emerging hybrid space produces an immersive teaching environment that acts as a learning modifier. More specifically, the HEPP infrastructure provides teachers and their pupils with the means to jointly create new educational resources as well as to re-design existing ones, either in the form of a structured course or a theatrical performance. The novelty in this approach is the possibility of visually layering, inside the classroom space, the information pertained to the educational material, by deconstructing its context and re-constructing it spatially into active and passive environmental layers of varying importance in the overall understanding of the subject presented. The second impetus driving this project is the establishment of a transcultural educational network between schools around the world. The co-operating schools work together and co-produce hybrid learning objects across multilingual educational communities worldwide. In the current state of the project, we focus on setting up a technology demonstration that includes all primary features: the interactive platform setting, a specific tele-learning content and a methodology for producing the analog objects and equipment which compliment the tele-projection between diverse locations and cultural settings. The objective is to help the pupils feel another place’s “genious loci” rather than simply learn facts by heart. Additionally, the project aims to blur the notions between inside and outside and virtually “teleport” the pupils by augmenting the perceived limits of the classroom’s environment.

Hybrid Environmenta l Pro j e c t ion Pla t form: An Interac t iv e Educat iona l Too l

Keywords

Interactive educational tool; augmented reality; visual layering; tele-learning; hybrid environmental projections.

Introduction

One of the main scopes of this paper is to illustrate the necessity for incorporating advanced technological systems, through design, inside the contemporary classroom space in order to create a new pedagogical tool that will actively enhance the learning process not only in terms of context, but most importantly, in terms of the learning experience. The stronger incentive that supports this direction is the image of the typical teacher-centered classroom that only envelops the educational activities without facilitating, either spatially or through equipment, the advancement of the learning process.

The Hybrid Environmental Projection Platform (HEPP) was conceptualized as a transcultural, interactive educational tool for participatory tele-learning. It constitutes a hybrid pedagogical tool that creates a customizable space of immersion in-between the actual and the virtual classroom and, furthermore, allows for the simultaneous 'presence' in diverse locations across cultures. In this context, the design of the educational space coincides with the design of the learning experience. Alongside, learning objects are co-designed and co-produced by both teachers and students.

The project is being developed at the Transformable Intelligent Environments Laboratory (TIE Lab) in the School of Architectural Engineering at the Technical University of Crete (TU Crete). TIE Lab is dedicated to the creation of environments that exhibit flexibility and adaptability through a human-centered, technology-driven architectural approach equipped with the appropriate design methods and tools, fabrication techniques, and implementation strategies. Moreover, all research projects in the Lab are multi-disciplinary, based in the field of architecture, and from there on expanding their scope within the domains of computer and electrical engineering, mechanical engineering, environmental engineering, pedagogy, psychology and neuroscience.

Multilingual and trans-cultural educational context for the Hybrid Environmental Projection Platform

The project is inspired by the Mobile Automatic Theatres in Ancient Greece, the work of Heron of Alexandria [1]. These automatons from antiquity were used to teach religious myths through portable interactive installations, through clockwork mechanisms and state-of-the-art, at the time, hydraulic and steam-powered systems. HEPP was conceptualised to provide a platform for the creation of educational and cultural events, from production to tele-projection, through the use of technological systems that facilitate tele-presence and immersion. The people involved in the original production of the event, design it in such a way so as to give the ability to their counterparts in another school to re-form the experience by re-directing scenes, by re-formulating educational objects based on local cultural traditions and by allowing for a variety of interactions to take place, through the proposed platform, between the projected event and the 'spectators'. The direct relationship of the HEPP with the distinct communicative ability of the produced events enhances the potential of employing, each time, a new immersive experience through creative metaphor from a variety of theatrical schools (Greco-roman, Elizabethan, French street theatre, Noh theatre, and so on). This potential lays the foundation to explore novel educational methods, especially regarding cultural themes.

At the current state of development, the HEPP infrastructure provides teachers and their pupils with the means to jointly create new educational resources as well as to re-design existing ones, either in the form of a theatrical performance or a structured course. The goal is to plan all the steps of the augmented learning experience and share it with other educational communities around the world. More analytically, this new paradigm for education has the ability to:

! Facilitate the co-operative development of learning objects, inspired by the cultural heritage of people and places.

! Promote multilingual educational networks that support tele-learning multimedia processes and, through that, globally communicate cultural heritage themes.

! Transmute spaces to act as active educational elements. ! Implement Information and Communication Technology (ICT) systems for the development of

immersive educational environments, where learning is received through multimodal stimuli and constructed through experience.

! Preserve local customs that hold special value to people and constitute a significant part of the ‘genius loci’ of a place, in the appropriate context.


! Understand culture and its production, through education. ! Support linguistic and cultural diversion. ! Promote the creation of multilingual educational networks with the participation of different countries,

languages and scientific fields. ! Promote educational equality by providing human resources and infrastructure that can support tele-

learning objects and processes that promote global education. ! Support the active interaction between regions for the development of new cultural objects and

traditions, creating a common cultural heritage. ! Negate cultural unfamiliarity, by bringing closer children and teachers from around the world,

empowering the notion of the "world cultural citizen." ! Support cultural heritage and education hubs.

In order to achieve the creation of an immersive environment, a simple projection would not suffice. A holistic experience must be technologically facilitated by creating environmental stimuli for more human perception mechanisms, providing that way adequate information for the brain to be immersed, just like being inside the projected place and time, experiencing the 'theme' live. The multimodal experience provided by the system allows for diversity in the ways a theme can be projected, all suitable for presenting different cultural assets.

The development of the HEPP

The project was initially conceptualized in two parts, the technical and the operational-educational. The latter involved issues regarding the design, process and representation of the information with which the educational content will be formed and 'transmitted' to another cultural setting. The technical part included the design of the setting and the requirements of the ICT systems, as well as the fabrication of the setting along with the assembly and operation of the embedded systems. The platform, as a complete system, is a hybrid digital-spatial installation that can operate synchronously and asynchronously, enabling the production of educational tele-learning objects to be projected to a network of educational communities.

Key Elements and Specifications of the HEPP plarform

The HEPP platform operates as a hybrid one, interweaving virtual and actual reality. It is able to convey a 'message', simply and precisely, in order to increase the potential for the 'recipient' to comprehend and relate with the meaning of the message. Moreover, the platform operates as a multi-agent system that allows “complexity through simplicity” and enables the formulation of diverse but distinct 'messages' in a multimodal way [2]. This function of the system is crucial for enhanced learning dynamics as it fortifies memorization and facilitates cooperation and understanding between multilingual participants. Furthermore, the HEPP platform hosts immersive educational events that address information to all the human senses, involving active spatial elements such as environmental projections, kinetic objects and partitions, targeted audio output, as well as devices that provoke haptic and olfactory stimuli. The multimodal experience increases the learning capacity of the pupils and adds value to the educational products, especially regarding cultural heritage themes. Scents, lighting and textures are types of environmental characteristics that can be tackled during the performance to reveal the feeling of a place and its cultural bearers while avoiding a stylized representation. The equipment that facilitates the augmented reality experience is adaptable and mobile in order to be actively distributed in various positions around the installation. The supporting tools allow interventions on characteristics of the projection, accommodating that way the interaction of cultures through the educational projects performed. Finally, the platform includes augmented connectivity features as its successful implementation is depended on operating through a network of educational communities that co-produce immersive educational events in multimodal editions.

The development of the platform follows three parallel directions: ! The design direction includes methods for designing and deploying the spatial arrangement of the HEPP

along with the projected themes that support each time the spatial experience. It also incorporates the pupil’s design interventions for each thematic setting. The design process produces methodologies, interior arrangements, objects, as well as the settings for the embedded system. In a way, the abilities of the platform as a whole are designed, either for the infrastructure required or the content to be projected. Especially for the content, the design part aims to provide the semiological tools required for the production of the individual cultural themes.


! The technical direction includes the platform development as infrastructure. It provides the ability to create the techno-spatial arrangements suitable for environmental projections, the robotic agents that operate around the space, the multi-modal projection system, the machine learning algorithms, the control software and the tele-directing software and system. Regarding tele-direction, the technical part includes also the utility software tools for the production of the themes [3].

! The educational direction includes the context that frames the production of the educational-cultural products. Pedagogical methodologies and practices, developmental psychology issues and multilingual communication strategies set the framework under the successful production and transmission of cultural heritage related educational material.

The first phase of development involved the design of the whole system of the infrastructure, addressing and fulfilling the aforementioned specifications and requirements. It is envisioned to operate as an adaptable multi-functional artefact. The whole installation is portable with the ability to unfold and get assembled on location. The dimensions of the installation fit within the typical classroom. Moreover, it can operate as an additional infrastructure element in larger rooms, such as auditoriums or school theatres, to provide additional possibilities for the production of school plays.

HEPP is designed to project information on two main spatial levels. The novelty in this approach is the possibility of visually layering the information pertained to the educational material, by deconstructing its context and re-constructing it spatially into active and passive environmental layers of varying importance in the overall understanding of the subject presented [4]. The distinction is directly linked with cognitive processes relevant to how information is perceived and understood by the human brain, facilitating that way optimum communication of information, messages and ideas. A general description would be that the passive layers project the sense of place, while the active layers project the actual information.

There are three passive layers responsible to provide the 'depth' of the experience. The last one in order, further back, creates the overall impression of the performance’s atmosphere, mostly by means of light, color and sound. The passive layer in the middle is employed to create spatial environmental references by placing landmarks in space that help the mind understand relative positions. The top passive layer projects information that, even though it is not directly related to the theme, plays a significant role on the way it will be perceived. Elements from this layer may change their status in the hierarchy and move up to the third active layer and vice-versa. The information inside the passive layers is discreet, manifesting through slow changes.

Along the same line of thought, there are three active layers comprised of the informational elements that are higher in the perceptual hierarchy and project the main figures of the educational object. The third active layer, the one adjacent to the first passive one, includes the elements that affect the theme in terms of how it is related to the environment and its main recognizable «keywords». The second active layer supports the narration and the understanding of the theme more significantly, while the first one includes the main figures and features that project actual information pertinent to the theme.

The system belongs to the category of cyber-physical objects having spatial elements embedded with informational systems. Assemblies of sensors, controllers and actuators allow a «tele-director» to have knowledge of the conditions under which the installation operates in order to control the behavior of the system accordingly. Specialized software will be further developed to assess the conditions of operation and provide 'creative consulting', through learning algorithms, and even direct interventions to maximize the impact of the platform’s goals. Furthermore, there is thought around the development of an application for mobile devices that will facilitate the process of getting, processing, directing and projecting the information, to allow for wider implementation and easier control.

Following is the technical description of the infrastructure that presents the initial development phase on setting up a working prototype with the main features, in order to be able to continually optimize the required software for operation and production of the content (please note that the exact design and materiality of the complete platform in the future may differ significantly):

! Structural framework – “Boxes”: The structural framework includes the basic elements needed to set up the standard HEPP assembly ensuring its stability, transportability and deployment ability on a selected site. It is comprised of stable parts upon which two types of kinetic elements are adjusted: those that provide the structure with additional adaptability and stability features within spaces of diverse characteristics and those that transform the stage’s characteristics between different scenarios. The mechanisms employed are also attached here. The structural framework is made of wood and specific metal and plastic pieces. They hold various “pockets” where the equipment can be fitted according to customizable requirements (Figure 1).


! Interactive robotic elements – “Bots”: These are the ‘protagonists,’ the automatons that form the necessary infrastructure to support the main performance’s interface with the spectators and guarantee its uniqueness. The Bots carry the active layers and have a more direct engagement with the spectators. They are autonomous and can avoid mishaps but tend to follow the movement of the distant narrator/actor and to present his/her mannerisms (Figure 2).

! Active performance elements - Interface: This part is the most essential one in terms of holding the performance, the director and the audience together by achieving targeted programmed reactions. In the pilot phase, there are a small number of such elements exhibiting a behavior representative to the whole.

! Electronic equipment and mechanisms: This is the heart of the functional as well as of the communicative part of the project. It is a combination of sensors for activity detection, of controllers for data management (UNO by Arduino and Galileo by Intel) and of actuators (small servo and stepper devices, LED lighting, microphones and so on), applied within any of the aforementioned parts at the will of the director. The pilot phase includes a small number of representative such parts.

! Software: This constitutes the system’s 'brain.' During the pilot phase studies have determined the research direction and the methodological approach. Its final development will take place during the next phases.

! Manufacturing infrastructure: It consists of specific units (Boxes) that both contain the technical equipment and provide storage for the Bots and the projection screens, so that the maximum ergonomy is achieved. They also help the user to assemble and disassemble the installation easier. The shape and the design of the Boxes leave space for adjustments and for a variety of spatial set ups that give the user the chance to experiment with the overall setting and create different experiences.

The first materialization phase has reduced complexity in order to facilitate the identification of occurring issues regarding the actual implementation and address them sufficiently. During the pilot phase, the setting is designed to support the equipment of two passive levels and one active. The Boxes and the Bots are designed and fabricated by assembling elements of transformable geometry, made by parametrically customized laser-cut wooden pieces and 3d-printed plastic pieces. They are fitted with the necessary equipment to perform according to the needs of the chosen event. The content of the performance was also selected as to be ideal for testing the basic operations of the system and provide clear feedback on adjustments and optimizations.

Progress so far

The project has now reached the level of technology demonstration in the field that will facilitate the validation of the overall logic and the identification of issues to be further addressed and optimized. It is the first step toward the development of the complete platform, through specific materialization steps that have been carefully planned to test the methods, the arrangements and the equipment of the project.


Figure 1: A visualization and a photograph of the Box in the demo version.

Figure 2: A visualization and a photograph of the Bot in the demo version.


Up to this point, the setting arrangement and its necessary spatial accessories have been fabricated according to the first educational site of implementation. The project is to be field tested for the first time within an elementary school classroom that is 6 by 10 meters in plan, with a specific setting due to the local regulations and the educational approach of the teachers. Certain restrictions led to a site-specific design calibration that differs from the original concept (Figure 3). Due to these restrictions (projection range, classroom dimensions) the initial square-shaped setting was converted into a diamond-shaped one, containing the necessary surfaces of the environmental projection. As a result, the Boxes that support the installation, equipped with two projectors each, were reduced to two and the remaining three projectors were placed in the surrounding area. This new arrangement, though, will enhance the multimodality and the depth of immersion into the experience. The projection surfaces are comprised of a double layering of fabrics (gauze and white cotton sheet with different density and transparency), amplifying the three-dimensional illusion of the optical effect conferred by the light refraction on the two fabrics.

Figure 3: The original HEPP spatial configuration (left) and the adapted one for the pilot study (right).

The classroom space has been divided into two parts to accommodate the performance. The first part, named 'recording space', is where the actual performance will be executed, while the second one, named 'playback space', is where the remote performance will take place. Motion from an actor on the recording space is captured and transmitted in real time on the playback space. The movement of the actor is re-played through a small robot that moves among the audience.

Regarding the systems embedded in the infrastructure, they address the different uses and implementation strategies decided for the two spaces. In the recording space, the setup consists of an area within which the actor moves, background images and a Microsoft Kinect sensor that is used to capture the movement of the actor. It is important to note that in the current setup the recording space will not be implemented. The robots’ movement is being manually controlled. The playback space, on the other hand, consists of 3 to 6 projectors that are used for the various environmental projections, 2 Boxes that are used as standing points for the screens and a small autonomous Bot. The Bot is moving among the audience with a 3d projection system on top of it. This certain technique is proposed by Bao and Xiang in their 3D Projection on Pyramid project [5]. The Bot is custom designed and manufactured by the HEPP team. The basic structural material used for the Bot is wood. It consists of 3 vertical levels, with every level being responsible for a separate function. The lower level contains the Arduino microcontroller that is used for navigating the Bot. It also has two motors connected to wheels for its movement and one IR distance sensor in front of it. This way the Bot is able to detect dynamically potential obstacles and avoid them. The second layer contains speakers that play the voice of the actor and the top-level houses the 3d projection pyramid.


The Boxes are designed so that they have enough space on their base to hold the Bot. This way, when the Bot is not moving, it can be stored there, hidden from the audience. Also they have openings for projectors to be inserted. This helps provide more flexibility in different set-ups. Finally, the sides of the Boxes are used as anchor points for the screens of the environmental projection. In the current setting, the projectors are placed in the surrounding space, outside the playback space.

The final parameter of the demo performance is the design and production of the content, which was made in regard to the test purposes and the known audience comprised by pupils in the 3rd grade of Elementary school. The scenario, which concerns an old Greek myth with the fox and the grapes, has been developed in order to highlight the potential of the prototype. The visual layering was carefully constructed to create a three-dimensional ambient environment that will enhance the audience’s immersion (Figure 4). The infrastructure elements will surround the children to increase the feeling that the audience is also a part of the performance. As the audience’s absorption in the performance is a goal, both video and audio clips (including the storyteller, music and ambient sounds of the forest) complete the background elements.

Figure 4: Various scenes from the demo performance. The video can be found here: https://www.youtube.com/watch?v=Pm4A9tR7lRM


The protagonist of the performance, the Bot, is a very specific, interactive element that wanders through the audience while performing and changes its route when obstacles appear. Moreover, it further highlights the fact that the audience is part of the show and participates in it as the script evolves. The role of the main character has already been recorded, in order to be transmitted, via the central speaker of the Bot. In addition, an actor has already been filmed in order to be shown through a pico projector mounted on the Bot. A dry ice machine will create a smoke surface, which will actually form the projection’s screen to give a dramatic effect on the main role (Figure 5).

Figure 5: Snapshots from the video production, experimenting on the actor’s appearance.

After several experiments with a smoke machine, it has been decided that the dry ice machine is the most reliable solution, mainly because the thickness of the dry ice projects better the emitted light. A second approach, that of creating a floating hologram on top of the Bot is also under consideration and will be field tested soon. As for the background, several different implementations have been fabricated and tested, in order to achieve adaptability in various environments. For the field testing, the background consists of two customized wooden structures containing projectors, background speakers, lighting equipment and store room for the Bot. Finally regarding, the flexibility, portability and deployment of the setting, two types of constructions with different assembly methodologies are tested. The first one consists of joints and the other one of screws. An important issue is the safe transportation and storage of the boxes and the adaptability into spaces with diverse characteristics.

Conclusion

In contemporary pedagogical approaches, the use of a/synchronous educational platforms that operate 'live' from different, distant and/or culturally diverse places, open up new possibilities for the emergence of multilingual, collaborative school communities. The key concept lies upon the co-operation between teachers and pupils as producers of new educational material through the creative transformation of knowledge bits into environmental and visual information. As part of the learning process, both groups participate in the creation of hybrid, interconnected environments that teach not only facts but also experiences. Active educational communities can significantly improve their pedagogical processes and, through that, help children develop their developmental skills and abilities. They will also become a critical innovation factor for the fostering educational system, and consequently a drive for societal development [6].

Another significant aspect of the proposal is evident in its potential to help educators revise and enhance the services they provide. The active involvement of the pupils, the 'out-of-the-box' way of thinking along with the creative use of knowledge and technology inside the classroom, directly lead to the improvement of the educators as professionals [7]. As a result, they receive recognition for their valued work that leads to an increase in their social status and "prestige" [8]. Their active participation in international learning communities also has a significant impact upon their work [9]. Furthermore, distinction, novelty, interaction and cultural equality become feasible as educational qualities through the tele-projection of cultural heritage themes with the use of multilingual, participatory and interactive educational tools. The proposed platform promotes open educational resources and open learning processes by providing useful and understandable means of expression and communication. Global educational resources can be more efficiently implemented through such a platform that allows the production of educational objects from a distance, providing at the same time an enhanced potential to transmit cultural messages.


At the current state, the HEPP project has entered its development phase and experimented with different spatial parameters and settings, with the “hybrid environmental projection method” considered to be the most dependable one. The use of conventional equipment was adequate but will be deficient for the next steps and a more advanced technology is required. Short throw projectors and advanced visual software are just around the corner, as we develop more specific and precise tools. Overall, the main research goal was to create an immersive experience based on flexibility and adaptability between the users that enhance the audience’s creativity and perception. A part of this has been accomplished, although experiments with professional equipment remain to be accomplished. The overall effect exhibits the outcome of the experiential design and the test provided useful data for the immediate follow-up of the HEPP development toward the complete platform.

Acknowledgements HEPP is developed as a major element toward the realization of the Automatic Mobile Theatre, a cultural-

education project, designed at the TU Crete TIE Lab in 2011 by J. Tzoumerkas, V. Petropoulou, K.-A. Oungrinis, M. Liapi, E. Ackermann, G. Andresakis, N. Asimakis, K. Hodaj, G. Inglezakis, M. Kantarzis, D. Mairopoulos and S. Vlachiotis.

Moreover, our thanks go to NEA TV, the local TV channel in Chania, Crete, that is our communication sponsor.

References

[1] Association of Ancient Greek Technology Studies (EMAET). 2000. Ancient Greek Te§chnology, 58. Technical Museum of Thessaloniki, Thessaloniki.

[2] Flavio Soares Correa da Silva and Wamberto Weber Vasconcelos. 2005. Agent-Based Management of Responsive Environments, Technical Report AUCS-TR0502, Department of Computing Science, University of Aberdeen, UK. Retrieved from http://inf.abdn.ac.uk/research/listpublications.php?year=2005&listing=techreports&report_no=AUCS/TR0502 (April 25, 2013).

[3] A. Paraschos, N. Spanoudakis, and M. Lagoudakis. 2012. Model-Driven Behavior Specification for Robotic Teams. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems, Valencia, Spain.

[4] M. Liapi. 2005. Spatial Diagnosis and Media Treatments. MIT SMArchS Thesis, MIT Department of Architecture, Cambridge, MA.

[5] Jiushi Bao and Mike Xiang. 2013. 3D Projection of Pyramid, Final Report for ECE 445. Senior Design, University of Illinois at Urbana-Champaign, Urbana, IL. Retrieved from http://courses.engr.illinois.edu/ece445/projects/fall2013/project36_final_paper.pdf (April 25, 2013).

[6] E. A. Hanushek and L. Woessmann. 2009. Do better schools lead to more growth? Cognitive skills, economic outcomes, and causation. NBER Working Paper No. 14633, Cambridge, MA, National Bureau of Economic Research.

[7] A. Barkatsas. 1999. The development of effective professional development programs for teachers: insights from international models, Mentor: A. In Journal of Scientific and Educational Research [P], Vol. 1, pages 76-91. Hellenic Pedagogical Institute, Athens.

[8] J. Elliott. 1991. A model of professionalism and its implication for teacher education. In British Educational Research Journal, 17 (4), pages 309-18.

[9] R. S. Prawat. 1996. Constructivisms, modern and postmodern. In Educational Psychologist, 31 (3/4), pages 215-225.

Hybrid Environmental Projection Platform: An Interactive Educational Tool

Documents

Transcript of Hybrid Environmental Projection Platform: An Interactive Educational Tool