The impact of Wireless {radio} as a medium for changing the way people think and create Ian A Hynd. MScR Digital Media and Culture: s1163663

Abstract A discussion on the scenario where ubiquitous wireless connectivity to handheld devices allows the creation of virtual worlds superimposed upon the real, and where the digital environment can become a marketing tool. What are the likely outcomes of this scenario? And how will they change the way we see the world? The use of geospatial data in gaming and art will be explored to get an insight into how the technique may evolve under political or commercial pressures.

2

The impact of Wireless {radio} as a medium for changing the way people think and create Ian A Hynd. MScR Digital Media and Culture: Edinburgh College of Art / Edinburgh University. December 2011

3

Content

INTRODUCTION............................................................................................................................. 4

AIMS & SCOPE................................................................................................................................. 4

REASON............................................................................................................................................ 4

CONSTRAINTS .................................................................................................................................. 4

TERMINOLOGY................................................................................................................................. 5

1: WIRELESS & AUGMENTED REALITY............................................................................... 5

LAYAR B.V. AMSTERDAM .............................................................................................................. 5

BUSHWICK AUGMENTED REALITY INTERVENTION 2010 .............................................................. 6

GOOGLE STREET VIEW.................................................................................................................... 8

2: OWNERSHIP................................................................................................................................ 8

3: CENSORSHIP & TRUST.........................................................................................................10

4: SUMMARY .................................................................................................................................11

REFERENCES ................................................................................................................................12

BIBLIOGRAPHY ..............................................................................................................................12

WEB LINKS .....................................................................................................................................12

REPORTS.........................................................................................................................................12

FILMS .............................................................................................................................................12

4

Introduction “The Man Who Sold the Moon” is a science fiction novella, (Heinlein, R. A. 1949) which describes the background and processes of a fictional moon landing in 1978. Within this novel a scenario is discussed where the protagonist raises the funding for the mission by speculating on the possibilities of using the moon as a giant advertising billboard. He proposes this by targeting the moons surface with missiles, carrying black dust which mark out patterns, altering the moons albedo and making logos visible. The protagonist raises funding by playing companies such as “Moka Cola” and “6+” against each other to not fire the missiles.

Aims & Scope This scenario, using the moon as an advertising billboard was ridiculous at the time, may now be on the verge of realisation via geolocation and wireless technologies. Handheld devices such as smart phones, GPS, digital cameras and tablet computers enabled with these technologies allow the creation of virtual worlds which can be seen as superimposed upon the real, which viewed via the device or its output and where the digital environment can utilised as a marketing tool. Is it realistic that when we do something as simple as take a digital photograph of, say, the moon, the image recorded is branded with a sponsor’s logo, theme or advert for a commercial company or public organisation. In this paper I intend to explore this scenario and it’s implications in relation to two manifestations of alternate or augmented reality, Layar & Google streetview.

Reason Various commercial companies have explored this area, recently we have seen a number Bluetooth RF based devices which broadcast adverts to mobile phone owners within the range of the device. However these devices fall within the remit of “The Electronic Commerce (EC Directive) Regulations 2002” and in the US the “Telephone Consumer Protection Act (TCPA), broadly speaking these acts prohibit the sending of unsolicited adverts to mobile phones, though as discussed by Brunar & Kumar (2007)

“the reality is that due to legal loopholes and lax enforcement, receiving commercial messages from parties to whom consent has not been explicitly given will be a fact of life for the near term”

Also in films such as Minority Report (Spielberg 2002), we see advertising tailored to the individual via billboards using facial recognition software, and though broadly similar to Bluetooth delivery systems in delivering and advert, the implied facial recognition application would be out-with the regulatory frameworks currently in place. Augmented reality does not seem to be bound to any regulatory framework or code of conduct what are the potential issues? Particularly those associated with ownership and censorship.

Constraints In considering this subject I envisaged looking at gaming in greater detail, however due to space constraints I have omitted this section, Geo-spatial data combined with fast high bandwidth wireless communication incorporated into mobile devices with high definition display screen and stereo audio has opened up enormous potential. Games such as Savannna (NESTA Futurelabs), Can You See Me Now ( Blast Labs) and Snowboard Hero (Layar VB) demonstrate the potential and I may revisit this in the future.

5

Terminology GPS, Global positioning system. A system for accurately determining position in 3 dimensions, currently operated by US Department of defence. A GPS receiver calculates its position by precisely timing the signals sent by GPS satellites which are in precisely known orbits. 3G & 4G Third & Fourth generation mobile telecommunications, wireless telephony systems capable of data rates of 200 kbit/s (3G) 1 Gbit/s (4G) both systems operate on a cellular system. the geographical location of a mobile phone can be determined using a technique known multilateration to calculate the differences in time for a signal to travel from the cell phone to each of several cell towers near the owner of the phone WiFi, IEEE 802.11 is a set of standards for implementing wireless local area network (WLAN) computer communication in the 2.4, 3.6 and 5 GHz frequency bands with data rates of up to 600 Mbit/s.

1: Wireless & Augmented Reality In considering what effect wireless radio has within a population it is important to distinguish between the forms of wireless communication which exist and analyse the impact they have on the group and individual. Radio (radiotelegraphy) as a broadcast medium has been in use since the early part of the twentieth century with the first commercial broadcasts around 1920, its ability and use to shape ideas and events is well documented (Brown 1963). Consider the impact of wireless communication networks, particularly those referred to as Wireless Wide Area Networks (wireless WAN), on interaction within a society. This technology, implemented by either 3G / 4G telephone networks or 802.11 node networks, enables an unprecedented level of information to be made available to the individual on a continuous, on demand basis. The expectation is that there will be a virtually unlimited data resource at the users fingertips, delivered via broadcast media and the internet with unfettered communication with other individuals or groupings of individuals. From a creative position it has been seen that we can use this wireless environment for the creation of alternative (augmented) realities where the use of wireless devices such as iphones, cameras, psp, ipad etc in conjunction with geospatial data, determined from GPS, 3/4G, & WIFI portals, can enable the manipulation of how we see and interact with our environment (Scharl 2007) (Matyas 2007), to enable digital environments to impinge on reality, in effect imposing a digital layer upon the real world. We can already interact to some extent with these augmented realities, by accessing Google Earth and Streetview we can to a certain extent experience the world through the computer screens. Mobile devices now have the power and resolution to bring a different dimension to our reality.

Layar B.V. Amsterdam Consider one of the market leaders in augmented reality systems, Layar B.V. Amsterdam, The Netherlands. (http://www.layar.com/) This company has developed a range of products focused on implementing augmented reality in the fields of Art, Gaming and Advertising utilising a number of mobile devices such as Android & IPhones, IPad etc.

6

“Layar works by using a combination of the mobile phone’s camera, GPS, compass, accelerometer and a mobile Internet connection. The camera captures the world as seen through its lens and shows it on the screen. The GPS determines the exact location and the compass and accelerometer determine the direction of view. Based on these sensors and the selected layer, digital information is retrieved over a mobile Internet connection and augmented on top of the camera view.”( Layer.com 2011)

We can see Layer’s potential for marketing in the illustration below, Here the application identifies a poster in the real world advertising an event and invites the viewer to purchase tickets online. We can see from the illustration superimposing the camera image with the virtual advert, how the virtual is placed over the real.

Of course this application and other like it are only by consensual use, clearly you would only see the advert above if you actively sought it out,

Bushwick Augmented Reality Intervention 2010 http://bushwickarintervention.wordpress.com/

This is an augmented reality art installation in Bushwick, Brooklyn New York. A number of artists were invited to create 3D digital artworks which were placed around the district. Visitors were invited to download an application (http://www.layar.com/player/) to their smart phones which allowed them to experience the artworks as they moved around the neighbourhood. The artworks included animations as well as static 3D images which are stitched onto / into real time images produced by the phones camera

Mark Skwarek Yuichi Irimoto

7

Tamiko Thiel John Craig Freeman

This event utilised the “Layar Reality BrowserTM” this is an application which is downloadable to a smart phone by those wishing to participate in the event. The combination of images and digital art is quite sophisticated; the images are three dimensional and highly detailed. But consider the image below.

This image, “Ghost Busters” by Han-Chun Yang, is the alteration of the real property in Brooklyn used for external shoots for the film Ghostbusters (Reitman, I. 1984). It is a humorous piece, a homage to the film, with the building slightly modified and artifacts from the movie digitally added. However it serves to illustrate how the virtual world can modify the real world with meaning / function changed, not necessarily with the real world owners consent.

8

Google Street View http://maps.google.co.uk/intl/en/help/maps/streetview/ This popular application, an adjunct to the Google Earth and Google Maps application, is particularly effective on mobile devices where it effectively provides a 3D representation of the real world for a point in time. It differs from the Layar application in the key aspect that it is not a live representation of the world. I include it in this discussion for several reasons, firstly it does offer a view of the world and secondly it can be augmented. Consider this Streetview image of Times Square New York. The image has been processed such that all forms of personal identification, faces and vehicle licence plates have been blurred. But note the bill boards in the upper half of the picture , the faces of the models have also been blurred but the text is unchanged So what we see through streetview is a highly edited world where some things are erased. More over this is an automatic process with a form of AI analysing the image and making decisions based on an algorithm. The viewer is looking at an image of the world which may not be representative of a point in time but may be the product of a continuous revision process. Consider the billboards for instance, it may seem reasonable to replace out of date signs with those of current shows on an ongoing basis, but if the original advert was on behalf of Theatre ‘A” would it be ethical to replace it with one for Theatre ‘B” also would it be reasonable to say replace a birth control advert with a pro-life advert? Further, consider the other blurred out elements within the image what if some faces and registrations were left un-blurred? What if faces and registrations were changed? Are there any controls on this? (Rogerson 2009)

2: Ownership In respect of my opening scenario, Heinlein’s proposal for advertising on the moon was based on the property ownership principle:

“Cuius est solum, eius est usque ad coelum et ad inferos - whoever owns [the] soil, [it] is theirs all the way [up] to Heaven and [down] to Hell” Accorso di Bagnolo; c. 1182 – 1263)

This principle granted ownership rights to landowners which gave legal rights to the space above and below their property to an infinite height above and to the centre of the earth below. Thus by purchasing land directly under the moons orbit the protagonist purchased a share of the moon, entitling him to place signage on it. Cuius est solum was invalidated by the advent of air travel and the moons status as “common to mankind” was enshrined by the “Treaty on Principles Governing the Activities of States in the Exploration and Use of Outer Space, including the Moon and

9

Other Celestial Bodies” (1967). Where does ownership rest within a geo-location based augmented reality? In the real world owners only have rights to airspace that they could reasonably use. It would be impractical from the development of air travel for individual landowners to own all the air above them, because airplanes would be constantly trespassing. The case can be argued that in augmented reality should property owners have rights commensurate with the real world? Again clearly it is not unreasonable to imagine the case where an individual or corporate body could incur real damage from an augmented reality. The concepts of property ownership which have evolved over the last few hundred years are now lacking, with geo-special data linking the virtual world to the real world, the owner of the real world property may / will have no control over the virtual world incarnations, which may be counter to their interests. The capitalist idea of property ownership and rights are redundant in this context, the nearest comparison may be with “Robber Barons” of yesteryear, who appropriated the rights and assets that they wanted without consideration. Consider a recently lodged patent,

“PCT/CN08/01845, Google Inc, Systems, methods, and apparatuses including computer program products for generating a custom language model. In one implementation, a method is provided. The method includes receiving a collection of documents; clustering the documents into one or more clusters; generating a cluster vector for each cluster of the one or more clusters; generating a target vector associated with a target profile; comparing the target vector with each of the cluster vectors; selecting one or more of the one or more clusters based on the comparison; and generating a language model using documents from the one or more selected clusters.”

I deliberately show the full abstract, for in the structured form patents are phrased, it is difficult to extract the meaning of the ideas behind the patent. But further reading into the claims and method for the patent reveal that it is a method / technique for application within Google Streetview. This application allows for the identification of Signage and advertising within Streetview images and the replacement of these signs and adverts with others. i.e.

“The link can be associated with a property owner, for example the property owner which owns the physical property portrayed. The link can alternatively be associated with an advertiser who placed the highest bid on the image recognized within the region of interest (e.g., poster, billboard, banner, etc.). Any portion of the geographic display image in which the region of interest is located can be selectable (e.g., hot-linked). For example, the image of the coffee shop can be hot-linked to an advertisement for the coffee shop.”

“ it looks like Google could potentially identify some billboards and banners in Street View images and then replace these real-life billboards with virtual ads from the highest bidders.” (Lardinois 2010) This application clearly raises issues over copyright ownership. If you are the owner of a building or company you may not necessarily hold copyright for your property. (Mann 2010), If Google auctions the rights to advertise on the virtual image of your building you may not have any rights to the image of your property. This may cause problems if

10

for instance, McDonalds buy the rights from Google to advertise on Burger King images and vice versa. The same issues apply to “Layar” and similar applications. Taking the issue further, the matter may not be restricted to these virtual billboards. With objects and locations identifiable, the software developer is able to add a virtual layer to almost any object, identifying Automobiles, Landmarks, Trademarks etc There may ultimately be many versions of the virtual world sitting on the servers of various corporations, each similar but with a different unique view of the world, users picking and choosing which view is in harmony with their own ethical view of the world. It is likely that in time certain views will become predominant Creators of these virtual worlds / augmented realities do so for many and varied reasons, amusement, education, curiosity, necessity, wealth, we cannot trust that these will be for our benefit we have to apply a critical analysis to what we experience

3: Censorship & Trust At some point do we have to also consider censorship, currently viewing this manipulation of the environment is by consent, you have to sanction the concept to experience it, but will this always be the case? To a great extent issue has become one of ‘Trust’, where the individual has to make critical judgements in respect of the reliability of the source. The internet allows greater access to information than ever before but the transient nature of this information, news, articles and papers often being subject to continual revision, places constraints on the reliability much of what is available. Currently we can see how Google censors Streetview. But, as discussed above, Google may have the ability to manipulate the images, removing some details, replacing and adding others. But if we do not know the source material how can we make judgements as two the honesty of the image? Take for instance the simple process of taking a photograph or video, as discussed earlier in the introduction. In the last few years we have seen the emergence of sophisticated multifunction devices typified by the “Smartphone” which have become popular as recording devices which have proved pivotal in recent world events. Consider the events in the “Arab Spring”

“An equally significant concept worth defining is citizen journalism, which provides ordinary citizens the opportunity to document their own version of reality and tell their own side of the story. It is distinguished from professional journalism in that ordinary citizens use digital media tools to report on events on the ground, uploading text and videos directly to the Internet or feeding the information and videos to media outlets.” (Khamis & Vaughn 2011)

However if the recording devices had applications not dissimilar to layar built-in, could we trust what we see? would we necessarily know if and how the image was manipulated? Even single function devices such as Digital Cameras are being equipped with high speed internet access, (Samsung SH100 WiFi Digital Camera) the advantages accrued in simplifying download, printing and sharing photos made this inevitable, these devices can provide access to alternate realities, augmented reality rather than faithfully recording an instant in time and place. Much of how propaganda delivers it’s message is not by the use of the “Big Lie” but of a lot of small lies gradually changing attitudes and perceptions, introducing new metaphors. The potential of mobile devices as a tool for dissent was to a great extent ignored by the various authorities, their effect, as out lined by Khamis & Vaughn and in “Policing the Riots” (UK Police federation, 2011) now dictate that controlling and

11

understanding this mobile technology has now risen high on the political agenda and is likely to be under scrutiny with a view to establishing structures to control the media.

4: Summary

So long as it is able to deceive without injuring, that master of deception, the intellect, is free; Nietzsche F (1873)

I have explored aspects of the influence of wireless communication on society, I chose to examine the impact geospatial data would have when applied to wireless enabled devices. In his essay “On Truth and Lies in a Nonmoral Sense” Nietzsche argues that man struggles with reality and constructs concepts, lies and metaphor to exist and interact with others. In respect of the real world this is presented as a fragile concept

“What then is truth? A movable host of metaphors, metonymies, and; anthropomorphisms: in short, a sum of human relations which have been poetically and rhetorically intensified, transferred, and embellished, and which, after long usage, seem to a people to be fixed, canonical, and binding. Truths are illusions which we have forgotten are illusions- they are metaphors that have become worn out and have been drained of sensuous force, coins which have lost their embossing and are now considered as metal and no longer as coins.” Nietzsche F (1873)

But where does this leave us? I have discussed the use of wireless communication to mobile devices in respect of geo referencing and how these technologies combined are powerful factors which can influence the way we see the world. On one hand they allow the manipulation of reality as a means to explore the world and ourselves, to educate, entertain and amuse. But simultaneously they can distort and manipulate with the potential to mislead, misinform and deceive. Initially I examined the potential of commercial advertising in virtual environments, there would seem to be two main scenarios, firstly, where typified by the “Layar” application is where useful information and processes are laid upon the real world and accessed on demand. In this the user actively consents to this and can be selective in their interaction. The second is more evasive where aspects of the virtual world are concealed from the user, i.e. Adverts in Google Street, and masquerade as real world. Where the distinction between worlds is ambiguous It is the consensual use which leads to art, social interaction and commerce, the hidden processes are those which can lead to control, censorship and manipulation. But there are also questions of ‘Self’, is the virtual world which is coming into being the playground of the few, or a basic right for all. There is a situation where the potential exists for unethical use of virtual worlds, exploitation and manipulation. Human rights have to be respected in augmented realities, processes and actions have to be transparent and accountable. Wireless technologies have already changed our world in terms of communication, today it is had to imagine people without mobile phones, it is expected that most people have access to mobile data. As the use of virtual and augmented realities starts to play an increasing role in our lives, we will have to expect the advertising industries to leverage these technologies, but the ethical issues discussed here, particularly ownership and political manipulation, will have to be addressed, safeguards designed and put in place to prevent misuse before we can trust this technology fully. Ian Hynd

12

References

Bibliography

1. Brown, J.A.C. Techniques of Persuasion: From Propaganda to Brainwashing Harmondsworth: Pelican (1963)

2. Bruner II, C. Kumar. A. Atitudes Towards Location-Based Advertising. Journal of interactive advertising spring 2007

3. Heinlein, R. A., The Man Who Sold the Moon. 1949 4. Khamis Dr Sahar & Vaughn, Katherine. Cyberactivism in the Egyptian Revolution: How Civic

Engagement and Citizen Journalism Tilted the Balance. Issue 13 of Arab Media and Society, Summer 2011

5. Lardinois. F. Google Plans to Upgrade Old Billboards in Street View. ReadWriteWeb, 2010 6. Mann, Phebe. Who owns the copyright of architectural works and designs? School of Construction

Management and Engineering, University of Reading, 2010 7. Matyas. Sebastian. Playful Geospatial Data Acquisition by Location-based Gaming Communities

The International Journal of Virtual Reality, 2007, 6(3):1-11 8. Rogerson Prof. Simon. Ethics of the Street “ ETHIcol” IMIS Journal Volume 19 No 3 (June 2009) 9. Sharl, Arno. 2007. Towards the geospatial web: Media platforms for managing geotagged

knowledge repositories. In A. Scharl, and K. Tochtermann (eds.), The Geospatial Web: How Geo-Browsers, Social Software and the Web 2.0 are Shaping the Network Society: 3-14. London: Springer.

10. Nietzsch,e Friedric.h On Truth and Lies in a Nonmoral Sense (1873)

Web links

11. http://www.readwriteweb.com/archives/google_could_soon_augment_old_billboards_in_street.php 12. http://www.layar.com 13. http://www.samsung.com/us/photography/digital-cameras/EC-SH100ZBPBUS

Reports

14. Police Federation of England and Wales. Policing the riots: response and recommendations (2011)

Films

15. Spilberg.S. Minortiy Report, Dreamworks, Amblin Entertainment. 2002 16. Reitman, I. Ghostbusters. Columbia Films.1984